-- Logs begin at Mon 2021-05-31 10:58:42 UTC, end at Mon 2021-05-31 12:17:31 UTC. -- May 31 11:46:03.441850 dv-cinder-791239-4-iscsi systemd[1]: Started Devstack devstack@q-svc.service. May 31 11:46:04.411636 dv-cinder-791239-4-iscsi neutron-server[98928]: Beware of the coming changes in the project packaging May 31 11:46:04.412252 dv-cinder-791239-4-iscsi neutron-server[98928]: https://github.com/svinota/pyroute2/discussions/786 May 31 11:46:05.886785 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.common.config [-] Logging enabled! May 31 11:46:05.887427 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev246 May 31 11:46:05.887543 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=98928) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 31 11:46:05.891548 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.common.config [-] Logging enabled! May 31 11:46:05.891801 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev246 May 31 11:46:05.892108 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=98928) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 31 11:46:05.893031 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 31 11:46:05.893265 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] Configuration options gathered from: {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 31 11:46:05.893653 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 31 11:46:05.893900 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 31 11:46:05.894123 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ================================================================================ {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 31 11:46:05.894441 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] allow_bulk = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.894734 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] allow_overlapping_ips = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.895025 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] api_extensions_path = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.895251 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] api_paste_config = api-paste.ini {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.895541 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] api_workers = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.895827 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] auth_strategy = keystone {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.896104 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] backlog = 4096 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.896395 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] base_mac = fa:16:3e:00:00:00 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.896685 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] bind_host = 0.0.0.0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.896965 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] bind_port = 9696 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.897243 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] client_socket_timeout = 900 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.897524 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] config_dir = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.897777 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.898053 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] config_source = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.898336 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] control_exchange = neutron {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.898618 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] core_plugin = ml2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.898838 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] debug = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.899110 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] default_availability_zones = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.899535 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.899950 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] dhcp_agent_notification = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.900227 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] dhcp_lease_duration = 86400 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.900512 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] dns_domain = openstacklocal {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.900788 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] enable_traditional_dhcp = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.901053 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] external_dns_driver = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.901325 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] filter_validation = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.901627 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] global_physnet_mtu = 1500 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.901949 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] host = dv-cinder-791239-4-iscsi {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.902221 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] http_retries = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.902501 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] instance_format = [instance: %(uuid)s] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.902781 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.903061 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ipam_driver = internal {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.903330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ipv6_pd_enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.903600 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] log_config_append = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.903879 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.904145 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] log_dir = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.904409 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] log_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.904679 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] log_rotate_interval = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.904959 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] log_rotate_interval_type = days {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.905260 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] log_rotation_type = none {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.905557 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.905801 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.906020 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.906310 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.906592 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.906863 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] max_dns_nameservers = 5 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.907132 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] max_header_line = 16384 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.907402 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] max_logfile_count = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.907671 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] max_logfile_size_mb = 200 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.907939 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] max_subnet_host_routes = 20 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.908203 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] network_link_prefix = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.908482 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] notify_nova_on_port_data_changes = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.908761 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] notify_nova_on_port_status_changes = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.909043 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] pagination_max_limit = -1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.909315 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] periodic_fuzzy_delay = 5 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.909605 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] periodic_interval = 40 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.909873 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] publish_errors = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.910139 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] rate_limit_burst = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.910419 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] rate_limit_except_level = CRITICAL {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.910687 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] rate_limit_interval = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.910953 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] retry_until_window = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.911222 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] rpc_response_max_timeout = 600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.911503 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] rpc_state_report_workers = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.911767 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] rpc_workers = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.912035 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] send_events_interval = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.912584 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.912863 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] setproctitle = on {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.913140 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] state_path = /opt/stack/data/neutron {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.913420 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] syslog_log_facility = LOG_USER {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.913786 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] tcp_keepidle = 600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.914067 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] transport_url = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.914336 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] use_eventlog = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.914604 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] use_journal = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.914869 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] use_json = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.915136 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] use_ssl = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.915402 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] use_stderr = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.915679 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] use_syslog = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.915948 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] vlan_transparent = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.916218 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] watch_log_file = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.916487 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] wsgi_default_pool_size = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.916754 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] wsgi_keep_alive = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.917036 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:05.917330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_concurrency.disable_process_locking = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.917645 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.917945 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.connection_string = messaging:// {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.918224 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.918516 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.es_doc_type = notification {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.918792 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.es_scroll_size = 10000 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.919077 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.es_scroll_time = 2m {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.919351 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.filter_error_trace = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.919635 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.hmac_keys = SECRET_KEY {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.919919 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.sentinel_service_name = mymaster {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.920204 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.socket_timeout = 0.1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.920478 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] profiler.trace_sqlalchemy = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.920761 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.enforce_new_defaults = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.921136 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.enforce_scope = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.921339 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.policy_default_rule = default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.921670 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.921964 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.922254 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.922527 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.922794 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.923064 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.923339 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.923624 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.923916 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.auth_type = password {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.924205 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.924475 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.924751 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.925039 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.endpoint_type = public {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.925315 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.925604 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.925882 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.live_migration_events = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.926172 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.region_name = RegionOne {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.926445 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.926790 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] nova.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.927128 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.927403 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.auth_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.927672 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.cafile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.927939 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.928212 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.928500 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.endpoint_type = public {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.928768 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.929034 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.929300 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.region_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.929593 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.929863 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] placement.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.930145 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.930418 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.auth_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.930687 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.cafile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.930957 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.931230 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.931502 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.connect_retries = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.931770 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.connect_retry_delay = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.932046 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.enable_notifications = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.932315 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.endpoint_override = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.932587 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.932855 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.interface = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.933125 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.933636 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.max_version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.934155 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.min_version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.934258 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.region_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.934528 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.service_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.934802 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.service_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.935080 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.935356 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.status_code_retries = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.935629 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.status_code_retry_delay = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.935903 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.936180 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.valid_interfaces = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.936456 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ironic.version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.937391 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] service_providers.service_provider = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.937725 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.938007 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.938291 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.938570 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.938848 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.939126 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.939402 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.939680 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.939959 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.940233 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.940525 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.940808 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.941093 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.941375 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.941674 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.941970 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.942246 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.942527 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.942801 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.943076 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.943362 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.943644 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.943938 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.944229 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.944520 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.944810 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_version = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.945100 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_notifications.driver = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.945377 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_notifications.retry = -1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.945712 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.945992 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] oslo_messaging_notifications.transport_url = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:05.946208 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 31 11:46:05.946458 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo.service.wsgi [-] Loading app neutron from /etc/neutron/api-paste.ini {{(pid=98928) load_app /usr/local/lib/python3.8/dist-packages/oslo_service/wsgi.py:349}} May 31 11:46:06.297246 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=98928) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:06.297452 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" released by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.001s {{(pid=98928) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:06.297739 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=98928) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:06.298218 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [-] Loading core plugin: ml2 May 31 11:46:06.690381 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.690709 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.691201 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.692789 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.693124 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.693467 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.728589 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.728866 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.729129 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.742082 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.742357 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.742618 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.772951 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.773235 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.773500 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.809703 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.809978 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:06.810245 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.043062 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.043675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.043675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment precommit_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.043843 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: network precommit_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.044509 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.169627 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.169951 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.170234 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.170509 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.170777 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.171608 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.171860 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.172114 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.172365 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.172644 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.172901 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network precommit_delete 0 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.173165 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port provisioning_complete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.173428 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.173719 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.173988 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.174257 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.175671 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete 0 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.175965 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.176235 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.176529 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.176889 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.177152 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.177406 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.177717 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.177974 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.178225 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.178612 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=98928) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 31 11:46:07.178858 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=98928) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 31 11:46:07.179098 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=98928) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 31 11:46:07.179334 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=98928) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 31 11:46:07.179569 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=98928) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 31 11:46:07.179803 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=98928) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 31 11:46:07.180168 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] May 31 11:46:07.183340 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['physnet1'] May 31 11:46:07.187885 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete May 31 11:46:07.191686 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('physnet1', [(350, 399)])]) May 31 11:46:07.193317 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan'] May 31 11:46:07.193793 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan']) May 31 11:46:07.194131 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['vlan'] May 31 11:46:07.194528 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security'] May 31 11:46:07.197468 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security'] May 31 11:46:07.197739 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security'] May 31 11:46:07.198084 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['openvswitch', 'hyperv'] May 31 11:46:07.225522 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.openvswitch.mech_driver.mech_openvswitch [-] Open vSwitch agent's supported_vnic_types: ['normal', 'direct', 'smart-nic', 'vdpa'] May 31 11:46:07.226006 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > qos_plugin after_init 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.226309 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.services.qos.drivers.openvswitch.driver [-] Open vSwitch QoS driver registered {{(pid=98928) register /opt/stack/neutron/neutron/services/qos/drivers/openvswitch/driver.py:82}} May 31 11:46:07.226676 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > logging-plugin after_init 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.228020 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.services.logapi.drivers.openvswitch.driver [-] Open vSwitch logging driver registered {{(pid=98928) register /opt/stack/neutron/neutron/services/logapi/drivers/openvswitch/driver.py:69}} May 31 11:46:07.228307 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['openvswitch', 'hyperv'] May 31 11:46:07.228556 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['openvswitch', 'hyperv'] May 31 11:46:07.231814 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_response 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.232100 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_response 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:07.305107 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat' May 31 11:46:07.305272 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete May 31 11:46:07.305496 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve' May 31 11:46:07.305901 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 1000)] May 31 11:46:07.336900 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_db.sqlalchemy.engines [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=98928) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 31 11:46:08.037702 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Initializing driver for type 'gre' May 31 11:46:08.038240 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] gre ID ranges: [(1, 1000)] May 31 11:46:08.305030 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Initializing driver for type 'local' May 31 11:46:08.305681 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Initializing driver for type 'vlan' May 31 11:46:08.431340 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] VlanTypeDriver initialization complete May 31 11:46:08.431683 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Initializing driver for type 'vxlan' May 31 11:46:08.432227 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] vxlan ID ranges: [(1, 1000)] May 31 11:46:08.745954 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Initializing extension driver 'port_security' May 31 11:46:08.746296 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.extensions.port_security [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] PortSecurityExtensionDriver initialization complete May 31 11:46:08.746625 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Initializing mechanism driver 'openvswitch' May 31 11:46:08.746949 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Initializing mechanism driver 'hyperv' May 31 11:46:08.754001 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.plugins.ml2.plugin [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=98928) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:46:08.754771 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > port after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.755144 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > port after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.755501 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > port after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.755887 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > subnet after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.756237 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > subnet after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.756580 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > subnet after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.756955 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > network after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.757332 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > network after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.757695 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > network after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.758070 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > security_group after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.758415 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > security_group after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.758754 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > security_group after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.759126 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > security_group_rule after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.759467 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > security_group_rule after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.759805 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > security_group_rule after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.760170 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > address_group after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.760515 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > address_group after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.760847 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > address_group after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.761179 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] ML2 OVO RPC backend initialized. {{(pid=98928) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:160}} May 31 11:46:08.761903 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router_interface after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.762253 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router_interface after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.762660 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > network before_response 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.763002 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > networks before_response 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.763346 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > port before_response 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.763681 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > ports before_response 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.764018 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > subnet before_response 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.764352 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > subnets before_response 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.765261 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > network after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.765631 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > network after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.765965 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > network after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.766299 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > port after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.766632 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > port after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.766964 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > port after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.767295 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > subnet after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.767623 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > subnet after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.767947 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > subnet after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.768518 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.plugin [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Modular L2 Plugin initialization complete May 31 11:46:08.768844 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading services supported by the core plugin {{(pid=98928) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:168}} May 31 11:46:08.769167 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.plugins.ml2.managers [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Got port-security extension from driver 'port_security' May 31 11:46:08.769649 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.extensions.vlantransparent [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Disabled vlantransparent extension. May 31 11:46:08.770137 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading service plugins: ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=98928) _load_service_plugins /opt/stack/neutron/neutron/manager.py:194}} May 31 11:46:08.770448 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading Plugin: router May 31 11:46:08.818942 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router_interface before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.819338 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router after_create 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.819708 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router before_create 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.820072 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router_interface before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.820441 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router_interface after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.820792 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router after_delete 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.821147 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > floatingip after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.821507 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router after_create 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.821880 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router after_update 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.822226 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router after_update 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.822571 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router_gateway after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.822924 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router after_update 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.823324 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_update 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.823686 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: port after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.824022 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: port after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.824356 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: subnet after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.824686 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: subnetpool_address_scope after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.825026 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > port precommit_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.825368 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_create 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.825739 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: port before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.826085 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.826440 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router after_update 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.826791 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_delete 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.827690 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_create 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.828048 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router_interface after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.828389 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_update 55549900 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.828756 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > network after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.829633 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Tracking information for resource: router configured {{(pid=98928) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 31 11:46:08.829963 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Tracking information for resource: floatingip configured {{(pid=98928) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 31 11:46:08.835012 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.db.l3_agentschedulers_db [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Skipping period L3 agent status check because automatic router rescheduling is disabled. May 31 11:46:08.835575 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: port after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.835937 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: port after_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.836288 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: port after_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.836641 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: port after_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.837433 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router before_create 55550100 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.837817 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_delete 55550100 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.838177 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_create 55550100 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.838526 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > router precommit_update 55550100 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.843132 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.services.provider_configuration [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service providers = [] {{(pid=98928) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:171}} May 31 11:46:08.843490 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.db.servicetype_db [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=98928) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}} May 31 11:46:08.846436 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.services.service_base [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded 'neutron.services.l3_router.service_providers.dvrha.DvrHaDriver' provider for service L3_ROUTER_NAT {{(pid=98928) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 31 11:46:08.846787 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.services.service_base [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded 'neutron.services.l3_router.service_providers.dvr.DvrDriver' provider for service L3_ROUTER_NAT {{(pid=98928) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 31 11:46:08.847127 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.services.service_base [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded 'neutron.services.l3_router.service_providers.ha.HaDriver' provider for service L3_ROUTER_NAT {{(pid=98928) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 31 11:46:08.848003 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.services.service_base [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded 'neutron.services.l3_router.service_providers.single_node.SingleNodeDriver' provider for service L3_ROUTER_NAT {{(pid=98928) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 31 11:46:08.848370 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.services.service_base [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Default provider is not specified for service type L3_ROUTER_NAT May 31 11:46:08.848923 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding between (L2) Neutron networks and access to external networks via a NAT gateway. {{(pid=98928) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 31 11:46:08.849244 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading Plugin: neutron.services.metering.metering_plugin.MeteringPlugin May 31 11:46:08.859734 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Successfully loaded METERING plugin. Description: METERING {{(pid=98928) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 31 11:46:08.860063 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading Plugin: auto_allocate May 31 11:46:08.872118 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: network precommit_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.872489 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: network precommit_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.873023 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=98928) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 31 11:46:08.873339 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading Plugin: tag May 31 11:46:08.880730 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=98928) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 31 11:46:08.881057 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading Plugin: timestamp May 31 11:46:08.884823 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=98928) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 31 11:46:08.885156 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading Plugin: network_ip_availability May 31 11:46:08.890486 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.890856 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.891210 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:08.891946 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=98928) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 31 11:46:08.892265 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading Plugin: flavors May 31 11:46:08.897328 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=98928) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 31 11:46:08.897685 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading Plugin: revisions May 31 11:46:08.907260 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=98928) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 31 11:46:08.907657 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_concurrency.lockutils [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Lock "manager" released by "neutron.manager.NeutronManager._create_instance" :: held 2.610s {{(pid=98928) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:08.908438 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] get_extension_paths = OrderedDict([('/opt/stack/neutron/neutron/extensions', 1)]) {{(pid=98928) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:635}} May 31 11:46:08.908787 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Initializing extension manager. May 31 11:46:08.909311 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: __init__.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.909698 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: __pycache__ {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.910161 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: _standard_attr_segment_lib.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.910495 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: address_group.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.911273 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.911845 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: address-group May 31 11:46:08.912429 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: address_scope.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.913132 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.913694 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: address-scope May 31 11:46:08.914015 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: admin_state_down_before_update.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.914980 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.915489 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: router-admin-state-down-before-update May 31 11:46:08.915838 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: agent.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.916530 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.917033 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: agent May 31 11:46:08.917350 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: agent_resources_synced.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.918269 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.918779 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: agent-resources-synced May 31 11:46:08.919099 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: allowedaddresspairs.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.921435 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.922025 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: allowed-address-pairs May 31 11:46:08.922347 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: auto_allocated_topology.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.923419 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.923931 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: auto-allocated-topology May 31 11:46:08.924254 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: availability_zone.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.924938 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.925441 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: availability_zone May 31 11:46:08.925787 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: availability_zone_filter.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.926684 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.927192 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: availability_zone_filter May 31 11:46:08.927514 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: data_plane_status.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.928402 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.928907 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension data-plane-status not supported by any of loaded plugins May 31 11:46:08.929224 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: default_subnetpools.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.930141 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.930644 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: default-subnetpools May 31 11:46:08.930959 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: dhcpagentscheduler.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.931767 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.932274 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: dhcp_agent_scheduler May 31 11:46:08.932590 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: dns.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.933580 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.934086 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension dns-integration not supported by any of loaded plugins May 31 11:46:08.934404 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: dns_domain_ports.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.935297 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.935795 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension dns-domain-ports not supported by any of loaded plugins May 31 11:46:08.936110 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: dvr.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.936809 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.937311 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: dvr May 31 11:46:08.937647 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: empty_string_filtering.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.938544 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.939134 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: empty-string-filtering May 31 11:46:08.939456 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.940346 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.940842 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins May 31 11:46:08.941158 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.942077 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.942617 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins May 31 11:46:08.942934 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: external_net.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.943820 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.944372 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: external-net May 31 11:46:08.944689 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: extra_dhcp_opt.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.945604 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.946117 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: extra_dhcp_opt May 31 11:46:08.946431 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: extraroute.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.947318 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.947820 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: extraroute May 31 11:46:08.948134 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: extraroute_atomic.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.949158 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.949691 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: extraroute-atomic May 31 11:46:08.950009 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: filter_validation.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.950662 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.951166 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: filter-validation May 31 11:46:08.951475 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: fip_pf_description.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.968114 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.968735 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins May 31 11:46:08.969062 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: fip_port_details.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.970013 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.970523 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: fip-port-details May 31 11:46:08.970845 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: flavors.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.972073 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.972596 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: flavors May 31 11:46:08.972940 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.974652 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.975170 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins May 31 11:46:08.975496 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: floatingip_pools.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.976190 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.976694 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: floatingip-pools May 31 11:46:08.977013 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: ip_allocation.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.977934 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.978434 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension ip_allocation not supported by any of loaded plugins May 31 11:46:08.978754 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.979643 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.980144 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: ip-substring-filtering May 31 11:46:08.980462 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: l2_adjacency.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.981364 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.981881 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension l2_adjacency not supported by any of loaded plugins May 31 11:46:08.982196 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: l3.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.983173 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.984278 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: router May 31 11:46:08.984632 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: l3_conntrack_helper.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.986982 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.987503 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension l3-conntrack-helper not supported by any of loaded plugins May 31 11:46:08.987829 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.988738 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.989250 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: ext-gw-mode May 31 11:46:08.989697 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.990620 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.991121 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: l3-ha May 31 11:46:08.991440 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: l3_flavors.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.992326 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.992825 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: l3-flavors May 31 11:46:08.993181 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.994092 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.994586 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: l3-port-ip-change-not-allowed May 31 11:46:08.994900 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: l3agentscheduler.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.996010 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.996461 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: l3_agent_scheduler May 31 11:46:08.996779 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: logging.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:08.998277 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:08.998790 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension logging not supported by any of loaded plugins May 31 11:46:08.999222 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: metering.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.000163 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.000743 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: metering May 31 11:46:09.001060 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.002182 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.002691 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: metering_source_and_destination_fields May 31 11:46:09.003009 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: multiprovidernet.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.004045 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.004554 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: multi-provider May 31 11:46:09.004876 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: netmtu.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.005793 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.006291 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: net-mtu May 31 11:46:09.006607 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: netmtu_writable.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.007502 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.008005 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: net-mtu-writable May 31 11:46:09.008319 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: network_availability_zone.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.008973 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.009475 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: network_availability_zone May 31 11:46:09.009812 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: network_ip_availability.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.010886 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.011390 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: network-ip-availability May 31 11:46:09.011703 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: network_segment_range.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.012490 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.012993 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension network-segment-range not supported by any of loaded plugins May 31 11:46:09.013304 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: pagination.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.014291 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.014667 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: pagination May 31 11:46:09.014980 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: port_device_profile.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.015869 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.016374 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: port-device-profile May 31 11:46:09.016690 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.017770 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.018272 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: port-mac-address-regenerate May 31 11:46:09.018589 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.019547 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.020047 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: port-numa-affinity-policy May 31 11:46:09.020364 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: port_resource_request.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.021242 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.021768 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension port-resource-request not supported by any of loaded plugins May 31 11:46:09.022090 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: portbindings.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.022994 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.023500 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: binding May 31 11:46:09.023821 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: portbindings_extended.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.025069 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.025586 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: binding-extended May 31 11:46:09.025919 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: portsecurity.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.026807 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.027308 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: port-security May 31 11:46:09.027624 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: project_id.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.028566 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.028888 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: project-id May 31 11:46:09.029202 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: providernet.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.030684 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.031186 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: provider May 31 11:46:09.031499 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.035282 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.035801 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos not supported by any of loaded plugins May 31 11:46:09.036126 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.037641 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.038150 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins May 31 11:46:09.038476 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.059237 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.059750 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins May 31 11:46:09.060068 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos_default.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.060961 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.061494 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos-default not supported by any of loaded plugins May 31 11:46:09.061846 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos_fip.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.062493 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.063053 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos-fip not supported by any of loaded plugins May 31 11:46:09.063459 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos_gateway_ip.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.064370 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.064872 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos-gateway-ip not supported by any of loaded plugins May 31 11:46:09.065189 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos_port_network_policy.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.066098 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.066596 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos-port-network-policy not supported by any of loaded plugins May 31 11:46:09.066911 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos_rule_type_details.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.067801 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.068300 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos-rule-type-details not supported by any of loaded plugins May 31 11:46:09.068615 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: qos_rules_alias.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.069673 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.070211 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension qos-rules-alias not supported by any of loaded plugins May 31 11:46:09.070525 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: quotasv2.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.071602 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per tenant" updated="2012-07-29T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.071926 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: quotas May 31 11:46:09.072240 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: quotasv2_detail.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.073893 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.074229 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: quota_details May 31 11:46:09.074555 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: rbac.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.075619 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control tenant access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.076138 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: rbac-policies May 31 11:46:09.076463 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: rbac_address_group.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.077635 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.078185 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: rbac-address-group May 31 11:46:09.078511 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: rbac_address_scope.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.079409 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.079918 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: rbac-address-scope May 31 11:46:09.080235 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: rbac_security_groups.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.081119 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.081662 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: rbac-security-groups May 31 11:46:09.081985 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: rbac_subnetpool.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.082868 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.083374 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: rbac-subnetpool May 31 11:46:09.083690 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: revisionifmatch.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.084591 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.085096 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: revision-if-match May 31 11:46:09.085415 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: revisions.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.086613 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.087119 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: standard-attr-revisions May 31 11:46:09.087439 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: router_availability_zone.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.088104 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.088607 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: router_availability_zone May 31 11:46:09.088921 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: routerservicetype.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.089844 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.090349 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension router-service-type not supported by any of loaded plugins May 31 11:46:09.090665 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.091563 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.092067 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: security-groups-normalized-cidr May 31 11:46:09.092382 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: security_groups_port_filtering.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.093834 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.094341 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: port-security-groups-filtering May 31 11:46:09.094656 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.095550 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.096053 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: security-groups-remote-address-group May 31 11:46:09.096368 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: securitygroup.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.105760 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.106287 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: security-group May 31 11:46:09.106611 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: segment.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.107390 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.107892 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension segment not supported by any of loaded plugins May 31 11:46:09.108213 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.109184 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.109716 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins May 31 11:46:09.110033 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: servicetype.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.111168 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.111676 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: service-type May 31 11:46:09.111997 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: sorting.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.112945 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.113272 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: sorting May 31 11:46:09.113607 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: standard_attr_segment.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.114497 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.114999 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension standard-attr-segment not supported by any of loaded plugins May 31 11:46:09.115315 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: standardattrdescription.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.115988 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.116483 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: standard-attr-description May 31 11:46:09.116797 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: stateful_security_group.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.117734 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.118236 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: stateful-security-group May 31 11:46:09.118554 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: stdattrs_common.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.119121 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py May 31 11:46:09.119435 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.120402 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.120910 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension subnet-dns-publish-fixed-ip not supported by any of loaded plugins May 31 11:46:09.121228 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: subnet_onboard.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.122334 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.122876 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: subnet_onboard May 31 11:46:09.123194 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.124202 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.124736 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins May 31 11:46:09.125053 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: subnet_service_types.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.126113 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.126631 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: subnet-service-types May 31 11:46:09.126952 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: subnetallocation.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.128012 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.128526 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: subnet_allocation May 31 11:46:09.128846 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.129501 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.130056 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: subnetpool-prefix-ops May 31 11:46:09.130372 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.131295 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.131794 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins May 31 11:46:09.132110 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: tagging.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.133225 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.133751 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: standard-attr-tag May 31 11:46:09.134078 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: timestamp.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.135260 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.135769 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loaded extension: standard-attr-timestamp May 31 11:46:09.136089 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: trunk.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.137093 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.137623 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension trunk not supported by any of loaded plugins May 31 11:46:09.137944 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: trunk_details.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.138835 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.139335 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension trunk-details not supported by any of loaded plugins May 31 11:46:09.140210 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: uplink_status_propagation.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.141102 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.141619 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension uplink-status-propagation not supported by any of loaded plugins May 31 11:46:09.141937 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Loading extension file: vlantransparent.py {{(pid=98928) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 31 11:46:09.142666 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=98928) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 31 11:46:09.143163 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extension vlan-transparent not supported by any of loaded plugins May 31 11:46:09.145669 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:network May 31 11:46:09.146405 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:subnet May 31 11:46:09.148300 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:subnetpool May 31 11:46:09.148912 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:port May 31 11:46:09.150303 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.150733 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service FLAVORS assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.151347 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.151781 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.152106 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:router May 31 11:46:09.152763 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:floatingip May 31 11:46:09.153456 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service METERING assigned prefix: /metering {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.154935 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:09.155899 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:09.157689 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=98928) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:46:09.157689 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of CountableResource for resource:rbac_policy May 31 11:46:09.157945 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:security_group May 31 11:46:09.158582 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:security_group_rule May 31 11:46:09.159314 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service CORE assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.159639 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:subnetpool May 31 11:46:09.159971 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] subnetpool is already registered May 31 11:46:09.160575 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service CORE assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.160898 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:subnetpool May 31 11:46:09.161219 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] subnetpool is already registered May 31 11:46:09.164062 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.164496 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service FLAVORS assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.165082 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.165497 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.165849 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:router May 31 11:46:09.166171 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] router is already registered May 31 11:46:09.166754 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:floatingip May 31 11:46:09.167076 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] floatingip is already registered May 31 11:46:09.167692 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service METERING assigned prefix: /metering {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.168364 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of CountableResource for resource:rbac_policy May 31 11:46:09.168687 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] rbac_policy is already registered May 31 11:46:09.169224 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:security_group May 31 11:46:09.169544 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] security_group is already registered May 31 11:46:09.170154 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:security_group_rule May 31 11:46:09.170487 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] security_group_rule is already registered May 31 11:46:09.171191 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service CORE assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.171513 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:subnetpool May 31 11:46:09.171830 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] subnetpool is already registered May 31 11:46:09.172467 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.v2.resource_helper [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Service CORE assigned prefix: {{(pid=98928) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 31 11:46:09.172787 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Creating instance of TrackedResource for resource:subnetpool May 31 11:46:09.173101 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING neutron.quota.resource_registry [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] subnetpool is already registered May 31 11:46:09.173760 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: extensions {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.175277 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: address-groups {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.176885 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: address-scopes {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.178112 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: agents {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.179262 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: auto-allocated-topology {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.180504 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: availability_zones {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.181838 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: dhcp-networks {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.183246 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: dhcp-agents {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.184656 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: routers {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.186542 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: flavors {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.187764 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: service_profiles {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.189103 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: next_providers {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.190680 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: service_profiles {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.192116 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: floatingip-pools {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.193307 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: routers {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.195132 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: floatingips {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.196309 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: l3-routers {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.197702 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: l3-agents {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.199083 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: metering-labels {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.200422 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: metering-label-rules {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.201720 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: network-ip-availabilities {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.203671 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: bindings {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.205318 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: quotas {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.206779 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: quotas {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.208169 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: rbac-policies {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.209453 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: security-groups {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.210678 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: security-group-rules {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.211894 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: service-providers {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.213106 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: subnetpools {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.215515 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: subnetpools {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.216950 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.219244 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.334408 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.336139 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.337940 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.340032 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.341777 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.343773 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.345500 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.347663 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron.api.extensions [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Extended resource: tags {{(pid=98928) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 31 11:46:09.353538 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING keystonemiddleware.auth_token [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 31 11:46:09.358845 dv-cinder-791239-4-iscsi neutron-server[98928]: WARNING keystonemiddleware.auth_token [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 31 11:46:09.367656 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_concurrency.lockutils [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Acquired lock "singleton_lock" {{(pid=98928) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:266}} May 31 11:46:09.368134 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_concurrency.lockutils [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Releasing lock "singleton_lock" {{(pid=98928) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:282}} May 31 11:46:09.368754 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO oslo_service.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Starting 2 workers May 31 11:46:09.373459 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Started child 99039 {{(pid=98928) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 31 11:46:09.378205 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Started child 99040 {{(pid=98928) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 31 11:46:09.378860 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:09.379799 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO neutron.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Neutron service started, listening on 0.0.0.0:9696 May 31 11:46:09.380342 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [-] (99039) wsgi starting up on http://0.0.0.0:9696 May 31 11:46:09.380443 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Notify callbacks [] for process, before_spawn {{(pid=98928) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:09.382705 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:09.383152 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO oslo_service.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Starting 1 workers May 31 11:46:09.384083 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [-] (99040) wsgi starting up on http://0.0.0.0:9696 May 31 11:46:09.388028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Started child 99041 {{(pid=98928) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 31 11:46:09.388533 dv-cinder-791239-4-iscsi neutron-server[98928]: INFO oslo_service.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Starting 1 workers May 31 11:46:09.391911 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] calling RpcWorker wait() {{(pid=99041) _wait /opt/stack/neutron/neutron/service.py:128}} May 31 11:46:09.392501 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] returning from RpcWorker wait() {{(pid=99041) _wait /opt/stack/neutron/neutron/service.py:135}} May 31 11:46:09.393619 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Started child 99042 {{(pid=98928) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 31 11:46:09.393836 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:09.394125 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [-] neutron.plugins.ml2.plugin.Ml2Plugin method start_rpc_listeners called with arguments () {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:46:09.394424 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG neutron_lib.callbacks.manager [None req-6ab65eb9-42e9-446e-95b3-72fe48359d27 None None] Notify callbacks [] for process, after_spawn {{(pid=98928) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:09.396388 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=98928) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 31 11:46:09.396594 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 31 11:46:09.396778 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 31 11:46:09.396991 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 31 11:46:09.397326 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 31 11:46:09.397487 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 31 11:46:09.398015 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.398283 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.398558 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.398810 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.399109 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.399586 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:09.400071 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.400071 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.400260 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.400403 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.400650 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.400816 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.400975 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:46:09.401078 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.401237 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.401478 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.401826 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.402094 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.402326 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.402591 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.402766 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.403029 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] config_source = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.403294 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.403536 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.403802 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] debug = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.404064 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.404550 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.404776 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.405019 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.405314 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.406126 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.406425 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.406626 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.406868 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.407099 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.407312 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.407570 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.407810 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.408041 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.408277 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.408532 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.408744 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.409229 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] host = dv-cinder-791239-4-iscsi {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.409471 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.409707 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.409950 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.410205 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.410446 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.410659 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.410912 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.411150 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.411409 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.411627 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.411866 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.412377 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.412596 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_dir = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.412834 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.413009 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_options = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.413257 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.413496 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.413784 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.413970 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.414134 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.414301 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.414547 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.414706 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.414957 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.415190 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.415401 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.415610 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.415847 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.416061 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.416326 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.416560 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.416802 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.417135 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.417281 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.417479 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.417726 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.417954 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.418183 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.418428 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.418638 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.418870 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.419106 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.419315 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.419524 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.419770 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.420000 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.420253 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.421200 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.421200 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.421200 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.421540 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.421799 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.422139 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.422524 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.422780 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.423052 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.423222 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.423462 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.423689 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.423904 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_journal = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.424145 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_json = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.424354 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.424579 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.424819 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.425028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.425240 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.425448 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.425681 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.425912 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.426335 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.426605 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.426931 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.427173 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.427433 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.427675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.427908 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.428147 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.428377 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.428605 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.428907 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.429129 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.429456 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.429669 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.429933 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.430238 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.430494 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.430780 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.431016 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.48/identity {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436028 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.436675 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437941 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437941 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.437941 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.438145 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.438296 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.438535 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.438785 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.439001 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.439213 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.439444 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.439658 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.439888 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.440103 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.440333 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.440544 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.440764 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.440992 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.441255 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.441494 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.441733 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.441969 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.442184 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.442416 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.442645 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.442860 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.443069 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.443301 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.443510 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.443717 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.443945 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.444154 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.444383 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.444613 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.444824 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.445711 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.445874 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.446087 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.446478 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.446478 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.446708 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.446957 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.447388 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.447514 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.447756 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.447974 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.448220 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.448461 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.448860 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.448960 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.449332 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.449440 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.449638 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.449885 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.450125 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.450343 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.450555 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.450853 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.451202 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.451457 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.451711 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.451963 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.452731 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.453004 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.453267 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.453496 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.453788 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.454032 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.454288 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.454507 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.454774 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.455011 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.455276 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.455568 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.455857 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.456114 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.456360 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.456587 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.456822 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.457047 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.457289 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.457529 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.457797 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.458012 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.458278 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.458562 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.458725 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.458933 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.459467 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.459901 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.460270 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.460401 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.460583 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.460853 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.461105 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.461341 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.461627 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.461870 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.462099 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.462339 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.462580 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.48/identity {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.462890 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.463032 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.463239 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.463452 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.463681 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.463902 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.464132 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.464356 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.464610 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.465885 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.465885 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.465885 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.465885 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.465885 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.466444 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.466444 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.466444 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.466759 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.466887 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.467009 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.467213 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.467587 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.467748 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.467909 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.468122 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.468395 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.468679 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.468877 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.469455 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.469455 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.469749 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.469749 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.470000 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.470202 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.470410 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.470639 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.470852 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.471065 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.471302 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.471541 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.471770 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.472017 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.472229 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.472546 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.473510 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.473510 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.473510 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.473510 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.474036 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.474036 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.474341 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.474341 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.474575 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.475429 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.475429 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.475429 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.475796 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.475918 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.476044 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.476217 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.476421 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.476626 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.477060 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.477199 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.477839 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.477839 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.477839 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.478230 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.478383 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.478517 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.478656 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.478933 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.479357 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.479357 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.479629 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.479754 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.480212 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.480365 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.481424 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.481424 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.481424 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.481424 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.481424 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.482093 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.482093 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.482093 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.482433 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.482558 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.482766 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.482941 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.483221 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.483475 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.483718 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.483976 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.484312 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.484539 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.484792 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.485041 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.485263 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.485496 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.485767 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.486070 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.486364 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.486639 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.486909 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.487188 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.487476 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:350:399'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.487713 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.487920 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.488228 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.488581 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.488854 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.489119 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.489360 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.489672 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.489911 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.490143 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.490356 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.490591 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.490820 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.491057 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.491286 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.491525 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.491736 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.491957 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.492188 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.492425 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.492653 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.492868 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.493084 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.493239 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 31 11:46:09.493816 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=98928) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 31 11:46:09.493975 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 31 11:46:09.494126 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 31 11:46:09.494292 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 31 11:46:09.494458 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 31 11:46:09.494628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 31 11:46:09.494799 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.494955 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.495111 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.495265 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.495421 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.495609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.495772 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.495929 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.496072 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.496227 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.496382 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.496536 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.496691 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.496845 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.497001 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.497156 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.497320 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.497479 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.497662 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.497819 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] config_source = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.497973 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.498127 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.498285 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] debug = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.499465 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.499465 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.499465 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.499465 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.499794 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.499794 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.499949 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.500027 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.500134 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.500285 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.500618 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.500762 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.500916 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.501069 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.501223 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.501380 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.501533 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.501703 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] host = dv-cinder-791239-4-iscsi {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.501852 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.501995 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.502146 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.502300 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.502454 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.502607 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.502833 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.503027 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.503253 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.503410 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.503550 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.503706 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.503858 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_dir = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.504011 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.504162 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_options = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.504316 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.504472 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.504624 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.504802 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.504960 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.505114 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.505269 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.505422 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.505609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.505751 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.505905 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.506058 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.506210 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.506362 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.506515 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.506670 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.506823 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.506976 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.507130 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.507380 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.507493 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.507655 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.507807 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.507960 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.508128 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.508255 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.508520 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.508759 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.508920 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.509074 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.509228 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.509383 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.509826 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.509826 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.509826 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.510220 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.510380 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.510538 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.510692 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.510845 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.510998 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.511152 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.511309 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.511583 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.511583 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_journal = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.511778 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_json = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.511887 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.512040 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.512194 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.512347 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.512499 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.512654 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.512806 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.512960 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 11:46:09.513501 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.513501 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.513501 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.513809 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.513809 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.513975 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.514055 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.514255 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.514346 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.514497 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.515220 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.515461 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.515678 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.515846 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.516027 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.516163 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.516330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.516457 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.516614 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.516772 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.516929 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.517087 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.517261 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.517418 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.517589 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.48/identity {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.517752 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.517909 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.518065 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.518223 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.518384 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.518539 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.518873 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.519032 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.519223 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.519350 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.519556 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.519699 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.519977 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.519977 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.520139 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.520229 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.520351 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.520552 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.520682 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.520813 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.520971 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.521128 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.521289 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.521446 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.522628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.522628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.522628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.522628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.522628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.522628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.522628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.522628 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523482 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523482 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523482 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523482 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523482 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523969 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523969 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523969 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.523969 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.524379 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.524379 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.524379 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.524778 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.524778 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.525095 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.525095 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.525095 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.525819 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.525819 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.525819 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.525819 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.526296 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.526296 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.526296 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.526296 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.526795 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.526795 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.526795 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.527009 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.527090 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.527168 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.527326 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.527469 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.527629 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.527784 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.527941 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.528097 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.528254 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.528411 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.528575 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.528707 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.528860 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.529016 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.529170 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.529326 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.529611 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.529733 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.529811 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.529943 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.530094 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.530281 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.530436 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.530591 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.530750 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.530918 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.531478 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.531478 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.531478 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.532042 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.532042 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.532042 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.532442 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.532442 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.532804 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.532804 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.533120 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.533120 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.533396 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.533396 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.533712 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.533712 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.533976 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.533976 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.534250 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.534250 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.534512 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.534648 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.534785 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.534922 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.535060 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.535209 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.535347 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.535492 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.535631 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.535767 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.535903 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.536041 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.48/identity {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.536177 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.536313 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.536449 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.536585 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.536735 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.536872 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.537025 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.537417 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.537557 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.537720 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.537857 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.538017 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.538164 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.538313 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.538452 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.538589 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.538749 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.538894 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.539052 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.539207 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.539358 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.539499 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.539645 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.539783 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.539919 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.540056 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.540216 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.540360 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.540519 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.540663 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.540822 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.541050 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.541210 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.541350 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.541495 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.541733 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.541733 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.542000 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.542137 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.542275 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.542424 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.543112 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.543228 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.543369 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.543519 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.543672 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.543986 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.544151 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.544341 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.544453 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.544599 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.544758 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.544912 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.545070 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.545225 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.545529 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.545884 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.546277 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.546445 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.546742 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.546742 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.546999 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.547136 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.547272 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.547403 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.547548 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.547686 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.547833 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.547971 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.548107 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.548244 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.548380 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.548561 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.548703 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.548842 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.549018 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.549161 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.549286 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.549372 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.549522 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.549728 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.549870 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.550018 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.550173 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.550330 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.550486 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.550641 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.550807 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.550966 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551125 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551269 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551435 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.551609 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.553596 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.553596 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.553596 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.553596 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.553596 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.553596 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.553596 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554084 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:350:399'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554084 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554084 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554467 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554467 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554467 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554826 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554826 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.554980 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.555100 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.555211 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.555332 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.555480 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.555636 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.555782 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.555942 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.556101 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.556254 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.556411 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.556567 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.556723 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.556880 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.557037 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.557199 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 11:46:09.557335 dv-cinder-791239-4-iscsi neutron-server[98928]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=98928) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 31 11:46:09.562546 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:46:09.620940 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 55566) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:09.629935 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [-] 10.222.0.48 "GET / HTTP/1.1" status: 200 len: 253 time: 0.0067990 May 31 11:46:20.879351 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-24646c48-594f-4c78-b460-9084b6e85598 None None] get_active_networks_info from dv-cinder-791239-4-iscsi {{(pid=99041) get_active_networks_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:158}} May 31 11:46:20.986032 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-24646c48-594f-4c78-b460-9084b6e85598 None None] No non-hosted networks {{(pid=99041) auto_schedule_networks /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:60}} May 31 11:46:20.996185 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.agentschedulers_db [None req-24646c48-594f-4c78-b460-9084b6e85598 None None] DHCP Agent not found on host dv-cinder-791239-4-iscsi {{(pid=99041) list_active_networks_on_active_dhcp_agent /opt/stack/neutron/neutron/db/agentschedulers_db.py:455}} May 31 11:46:21.123449 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3277bf80-7013-4853-afae-67afdb134de8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318'] for agent, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:23.334940 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318'] for agent, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:23.336996 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.337825 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.347840 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.347840 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.347840 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.347840 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.347840 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.348894 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.348894 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.348894 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.348894 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.348894 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:23.349782 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServersRpcApi method report_agent_resource_versions called with arguments (, 'Open vSwitch agent', 'dv-cinder-791239-4-iscsi', {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:46:23.364979 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServerRpcCallback method report_agent_resource_versions called with arguments (,) {'agent_type': 'Open vSwitch agent', 'agent_host': 'dv-cinder-791239-4-iscsi', 'version_map': {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:46:23.512419 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c2d9df4d-132f-4084-91e7-2ec0c38e5554 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318'] for agent, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:23.580913 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.db.api [None req-d62428c1-7980-4dd6-9aed-4250cad8287b None None] Retry wrapper got retriable exception: Failed to create a duplicate Agent: for attribute(s) ['agents.uniq_agents0agent_type0host'] with value(s) L3 agent-dv-cinder-791239-4-iscsi {{(pid=99041) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 31 11:46:23.581226 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_db.api [None req-d62428c1-7980-4dd6-9aed-4250cad8287b None None] Performing DB retry for function neutron.db.agents_db.AgentDbMixin.create_or_update_agent {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 31 11:46:24.179343 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-d62428c1-7980-4dd6-9aed-4250cad8287b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:24.614277 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318'] for agent, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:26.060044 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318'] for agent, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:26.568790 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.db.l3_hamode_db [None req-1b70ebdb-d093-4193-b006-b0e75de1c66c None None] Number of active agents lower than max_l3_agents_per_router. L3 agents available: 1 May 31 11:46:26.654637 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1b70ebdb-d093-4193-b006-b0e75de1c66c None None] Routers set() bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:46:26.655014 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1b70ebdb-d093-4193-b006-b0e75de1c66c None None] Router IDs set() for agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:464}} May 31 11:46:27.738138 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 55660) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:28.338891 dv-cinder-791239-4-iscsi neutron-server[99039]: WARNING oslo_policy.policy [None req-8baee78c-fdd6-44e2-99c5-2e6dbc4b7950 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 31 11:46:28.339499 dv-cinder-791239-4-iscsi neutron-server[99039]: WARNING oslo_policy.policy [None req-8baee78c-fdd6-44e2-99c5-2e6dbc4b7950 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 31 11:46:28.557430 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [None req-8baee78c-fdd6-44e2-99c5-2e6dbc4b7950 admin admin] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99039) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:46:28.558694 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8baee78c-fdd6-44e2-99c5-2e6dbc4b7950 admin admin] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.8185246 May 31 11:46:30.885200 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 55674) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:31.244528 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] Request body: {'subnetpool': {'prefixes': ['172.20.0.0/16'], 'shared': True, 'name': 'shared-default-subnetpool-v4', 'is_default': True, 'default_prefixlen': 26}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:46:31.344784 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.policy [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=99039) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 31 11:46:31.389743 dv-cinder-791239-4-iscsi neutron-server[99039]: WARNING oslo_policy.policy [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 31 11:46:31.390113 dv-cinder-791239-4-iscsi neutron-server[99039]: WARNING oslo_policy.policy [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 31 11:46:31.398441 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.quota [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] Loaded quota_driver: . May 31 11:46:31.413864 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:46:31.608672 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] Notify callbacks ['neutron.objects.securitygroup.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8759696086036', 'neutron.objects.address_group.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8759696086742', 'neutron.objects.network.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8759696091281', 'neutron.objects.qos.policy.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8759696069709', 'neutron.objects.subnetpool.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8759696076480', 'neutron.objects.address_scope.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8759702017830', 'neutron.plugins.ml2.plugin.External_net_db_mixin._process_ext_policy_create-1574274', 'neutron.plugins.ml2.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-1682688', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-651053', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-3588506'] for rbac-policy, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:32.086976 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] Notify callbacks [] for subnetpool, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:32.088160 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f7b585ac-4fe4-44c5-8b4b-3346c995852f admin admin] 10.222.0.48 "POST /v2.0/subnetpools HTTP/1.1" status: 201 len: 739 time: 1.2019358 May 31 11:46:34.662278 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 55688) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:35.251679 dv-cinder-791239-4-iscsi neutron-server[99040]: WARNING oslo_policy.policy [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 31 11:46:35.252263 dv-cinder-791239-4-iscsi neutron-server[99040]: WARNING oslo_policy.policy [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 31 11:46:35.466831 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Request body: {'network': {'project_id': '8be0fb13457d403886185c544e983ded', 'name': 'private', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'admin_state_up': True}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:46:35.468476 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.quota [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Loaded quota_driver: . May 31 11:46:35.483734 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:46:35.490559 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Usage tracker for resource:network and tenant:8be0fb13457d403886185c544e983ded is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:46:35.495956 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:46:35.503970 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:46:35.653703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Made reservation on behalf of 8be0fb13457d403886185c544e983ded for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:46:35.734912 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.policy [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=99040) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 31 11:46:35.782920 dv-cinder-791239-4-iscsi neutron-server[99040]: WARNING oslo_policy.policy [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 31 11:46:35.783308 dv-cinder-791239-4-iscsi neutron-server[99040]: WARNING oslo_policy.policy [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 31 11:46:35.794862 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:35.816267 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:35.830969 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:46:35.835018 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Usage tracker for resource:security_group_rule and tenant:8be0fb13457d403886185c544e983ded is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:46:35.839920 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:46:35.846142 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:46:35.942883 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:36.105384 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:36.111303 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:36.131757 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] vlan segment allocate from pool success with {'vlan_id': 384, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:46:36.156143 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8759697518309', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:36.156756 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Unsubscribe: segment precommit_create {{(pid=99040) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 31 11:46:36.157796 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Added segment ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee of type vlan for network e758f9f9-1a52-43f1-8476-bfac922bc884 May 31 11:46:36.166454 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=38535599-2c7d-46ba-80f1-a050f503a824,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:36.181883 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.182227 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.182546 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.182861 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.183167 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.183483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.183786 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.184084 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.184396 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.184694 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.184987 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.185282 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Tracked resource versions report: May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: pushed versions: May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: { 'AddressGroup': {'1.2'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'ConntrackHelper': {'1.0'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Log': {'1.0'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Network': {'1.1'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Port': {'1.7'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'PortForwarding': {'1.2'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'QosPolicy': {'1.8'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SecurityGroup': {'1.4'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SecurityGroupRule': {'1.2'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SubPort': {'1.0'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Subnet': {'1.1'}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Trunk': {'1.1'}} May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: consumer versions: May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: { DHCP agent@dv-cinder-791239-4-iscsi: {}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: L3 agent@dv-cinder-791239-4-iscsi: {}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: Metadata agent@dv-cinder-791239-4-iscsi: {}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: Metering agent@dv-cinder-791239-4-iscsi: {}, May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: Open vSwitch agent@dv-cinder-791239-4-iscsi: { 'AddressGroup': '1.2', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'ConntrackHelper': '1.0', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Log': '1.0', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Network': '1.1', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Port': '1.7', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'PortForwarding': '1.2', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'QosPolicy': '1.8', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SecurityGroup': '1.4', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SecurityGroupRule': '1.2', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SubPort': '1.0', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Subnet': '1.1', May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Trunk': '1.1'}} May 31 11:46:36.186319 dv-cinder-791239-4-iscsi neutron-server[99040]: {{(pid=99040) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 31 11:46:36.276463 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.165s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:36.283180 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:36.419548 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:36.421010 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:46:36.421680 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:36.426365 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:36.529036 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Pushing event updated for resources: {'Network': ['ID=e758f9f9-1a52-43f1-8476-bfac922bc884,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:36.532237 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:36.540518 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network e758f9f9-1a52-43f1-8476-bfac922bc884 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:46:36.597406 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 849 time: 1.9340506 May 31 11:46:39.203300 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 55710) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:39.646227 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0ffcf9e0-7596-4f5f-ab43-cef55bb94fe0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:46:39.647627 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-0ffcf9e0-7596-4f5f-ab43-cef55bb94fe0 admin admin] 10.222.0.48 "GET /v2.0/networks/e758f9f9-1a52-43f1-8476-bfac922bc884 HTTP/1.1" status: 200 len: 825 time: 0.4433587 May 31 11:46:39.678539 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-215018fa-4ff0-4e50-b0a7-2e099c163e67 admin admin] 10.222.0.48 "GET /v2.0/subnetpools/a90ae401-2f98-4796-953a-0a02d0edcb63 HTTP/1.1" status: 200 len: 734 time: 0.0255642 May 31 11:46:39.686806 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Request body: {'subnet': {'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'ip_version': 4, 'gateway_ip': '172.20.0.1', 'name': 'private-subnet', 'subnetpool_id': 'a90ae401-2f98-4796-953a-0a02d0edcb63'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:46:39.691153 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:46:39.695027 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Usage tracker for resource:subnet and tenant:8be0fb13457d403886185c544e983ded is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:46:39.697987 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:46:39.707120 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:46:39.830567 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Made reservation on behalf of 8be0fb13457d403886185c544e983ded for: {'subnet': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:46:39.832812 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Notify callbacks [] for subnet, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:40.350927 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:40.355763 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:40.431041 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Pushing event updated for resources: {'Subnet': ['ID=5ba76f57-f24a-4beb-8571-668a63f94cf9,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:40.432970 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Checking segment: {'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:46:40.433331 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Network hostable DHCP agents. Network: e758f9f9-1a52-43f1-8476-bfac922bc884, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99040) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:46:40.438502 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:40.543524 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Network e758f9f9-1a52-43f1-8476-bfac922bc884 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99040) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:46:40.651911 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, e758f9f9-1a52-43f1-8476-bfac922bc884) {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:46:40.656601 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:46:40.657188 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:40.661738 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-fbc1cc17-f844-49b5-9e16-9c61d47b4efb None None] Network e758f9f9-1a52-43f1-8476-bfac922bc884 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:46:40.788496 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-154171be-3982-4e32-b325-af908fb303da admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 854 time: 1.0954134 May 31 11:46:40.969751 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-e758f9f9-1a52-43f1-8476-bfac922bc884', 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'fixed_ips': [{'subnet_id': '5ba76f57-f24a-4beb-8571-668a63f94cf9'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 11:46:40.970927 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:41.157878 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Allocated IP 172.20.0.2 (e758f9f9-1a52-43f1-8476-bfac922bc884/5ba76f57-f24a-4beb-8571-668a63f94cf9/8eb0f362-8d0c-45b4-aaa8-4601b4e7de24) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:46:41.386025 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:41.398913 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Transition to ACTIVE for port object 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:46:41.418485 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Transition to ACTIVE for port object 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:46:41.527886 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:41.534345 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:41.599773 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Network e758f9f9-1a52-43f1-8476-bfac922bc884 is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:46:41.602816 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:46:41.606265 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:46:41.645360 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Pushing event updated for resources: {'Port': ['ID=8eb0f362-8d0c-45b4-aaa8-4601b4e7de24,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:41.690772 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.691410 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.691500 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.691818 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.692132 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.692471 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.692813 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.693150 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.693526 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.694159 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.694536 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.695076 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Tracked resource versions report: May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: pushed versions: May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: { 'AddressGroup': {'1.2'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'ConntrackHelper': {'1.0'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Log': {'1.0'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Network': {'1.1'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Port': {'1.7'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'PortForwarding': {'1.2'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'QosPolicy': {'1.8'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SecurityGroup': {'1.4'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SecurityGroupRule': {'1.2'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SubPort': {'1.0'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Subnet': {'1.1'}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Trunk': {'1.1'}} May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: consumer versions: May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: { DHCP agent@dv-cinder-791239-4-iscsi: {}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: L3 agent@dv-cinder-791239-4-iscsi: {}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: Metadata agent@dv-cinder-791239-4-iscsi: {}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: Metering agent@dv-cinder-791239-4-iscsi: {}, May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: Open vSwitch agent@dv-cinder-791239-4-iscsi: { 'AddressGroup': '1.2', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'ConntrackHelper': '1.0', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Log': '1.0', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Network': '1.1', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Port': '1.7', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'PortForwarding': '1.2', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'QosPolicy': '1.8', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SecurityGroup': '1.4', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SecurityGroupRule': '1.2', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SubPort': '1.0', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Subnet': '1.1', May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Trunk': '1.1'}} May 31 11:46:41.696129 dv-cinder-791239-4-iscsi neutron-server[99041]: {{(pid=99041) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 31 11:46:41.704565 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:41.720670 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Attempting to bind port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:46:41.721060 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Attempting to bind port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:46:41.721477 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Attempting to bind port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 on network e758f9f9-1a52-43f1-8476-bfac922bc884 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:46:41.726636 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 46, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:46:41.727037 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Checking segment: {'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:46:41.727477 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:41.727855 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Bound using segment: {'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:46:41.728363 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Bound port: 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:46:41.728737 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:41.817300 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] For port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:46:41.821078 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] For port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=8eb0f362-8d0c-45b4-aaa8-4601b4e7de24,segment=NetworkSegment(ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee),segment_id=ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:46:41.922907 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:41.927019 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:42.015651 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Pushing event updated for resources: {'Port': ['ID=8eb0f362-8d0c-45b4-aaa8-4601b4e7de24,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:42.019347 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:43.148988 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 55720) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:43.525240 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [None req-aadf074e-c59d-44d1-9b97-902adca5667c admin admin] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99040) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:46:43.526249 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-aadf074e-c59d-44d1-9b97-902adca5667c admin admin] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3762052 May 31 11:46:45.889934 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 55730) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:46.282735 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [None req-52f83c6b-0421-48f5-8d6f-8aae6cfb1730 admin admin] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99040) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:46:46.283707 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-52f83c6b-0421-48f5-8d6f-8aae6cfb1730 admin admin] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3927984 May 31 11:46:47.349929 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:46:47.464000 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-50af4c9b-b664-481c-acb4-69240b675de2 None None] Network e758f9f9-1a52-43f1-8476-bfac922bc884 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:46:47.527103 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:46:47.687636 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Provisioning for port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:46:47.698761 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Provisioning complete for port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:46:47.699159 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:47.839625 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:47.877067 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] For port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=8eb0f362-8d0c-45b4-aaa8-4601b4e7de24,segment=NetworkSegment(ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee),segment_id=ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:46:47.956396 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:47.959776 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:48.025839 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Pushing event updated for resources: {'Port': ['ID=8eb0f362-8d0c-45b4-aaa8-4601b4e7de24,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:48.029012 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:48.076193 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b513f5e-2f43-4af2-87ce-61d7ce0fb112 None None] Provisioning complete for port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:46:48.076481 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b513f5e-2f43-4af2-87ce-61d7ce0fb112 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:48.930186 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 55746) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:49.277518 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Request body: {'router': {'name': 'router1', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'admin_state_up': True}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:46:49.281955 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:46:49.286419 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Usage tracker for resource:router and tenant:8be0fb13457d403886185c544e983ded is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:46:49.290419 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:46:49.296564 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:46:49.334829 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:46:49.395315 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:49.397242 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:49.399269 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:46:49.399638 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:49.419255 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:49.602903 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:49.693610 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:49.694920 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-32a759b5-17b1-42ed-8ec7-787d6ae8ce0c admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 709 time: 0.7637665 May 31 11:46:50.904882 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-000f3611-e806-45e7-9bc5-39bb296a1f14 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:52.013939 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 55756) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:52.388369 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [None req-e0efe40e-76a8-4a33-97a8-36f084a1cd45 admin admin] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99039) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:46:52.389321 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-e0efe40e-76a8-4a33-97a8-36f084a1cd45 admin admin] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3744760 May 31 11:46:53.333402 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:53.516155 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ba7345bb-9165-4330-aca1-32144a47f261 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:54.531502 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:54.723422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 55766) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:55.087825 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Request body: {'network': {'provider:physical_network': 'physnet1', 'admin_state_up': True, 'provider:network_type': 'flat', 'router:external': True, 'name': 'public', 'is_default': True}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:46:55.092733 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:46:55.096486 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Usage tracker for resource:network and tenant:cf8b125186c446b2aec91a3f69c5f1fa is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:46:55.099313 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:46:55.104907 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:46:55.258994 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Made reservation on behalf of cf8b125186c446b2aec91a3f69c5f1fa for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:46:55.261190 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:55.272784 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:55.289959 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:46:55.293684 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Usage tracker for resource:security_group_rule and tenant:cf8b125186c446b2aec91a3f69c5f1fa is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:46:55.296371 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:46:55.302468 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:46:55.379704 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:55.493033 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:55.496180 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:55.518425 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.type_flat [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Reserving flat network on physical network physnet1 {{(pid=99040) reserve_provider_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_flat.py:100}} May 31 11:46:55.532361 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=982ee834-c738-46e8-a254-ae4dae7c9d8a,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:55.536139 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:55.541120 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:55.541455 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Added segment ec17477a-1229-4e63-8304-a2f0dbd731dd of type flat for network 8a73d753-6759-4d77-a99f-016e97b0a680 May 31 11:46:55.571219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:55.692902 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:55.694208 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:46:55.694774 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:55.699153 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:55.794880 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Pushing event updated for resources: {'Network': ['ID=8a73d753-6759-4d77-a99f-016e97b0a680,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:55.797417 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:46:55.804522 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 8a73d753-6759-4d77-a99f-016e97b0a680 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:46:55.886980 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 1.1624951 May 31 11:46:55.996318 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:58.278440 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 55786) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:46:58.486730 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-eb64bc14-ef86-4e5f-af2d-6901fb4017fe admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:46:58.488006 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-eb64bc14-ef86-4e5f-af2d-6901fb4017fe admin admin] 10.222.0.48 "GET /v2.0/subnets/5ba76f57-f24a-4beb-8571-668a63f94cf9 HTTP/1.1" status: 200 len: 849 time: 0.2086444 May 31 11:46:58.681649 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-34a27e40-ab05-4377-9a1a-bbfe6f275e09 admin admin] 10.222.0.48 "GET /v2.0/routers/b06623f8-649a-45fb-9b57-d4616c706968 HTTP/1.1" status: 200 len: 704 time: 0.1883132 May 31 11:46:59.144245 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:59.367385 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Allocated IP 172.20.0.1 (e758f9f9-1a52-43f1-8476-bfac922bc884/5ba76f57-f24a-4beb-8571-668a63f94cf9/a664371b-0c78-4e7b-9b16-cdeee995b663) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:46:59.530295 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:59.563418 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Transition to ACTIVE for port object a664371b-0c78-4e7b-9b16-cdeee995b663 will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:46:59.699887 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:59.704794 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:46:59.774970 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Network e758f9f9-1a52-43f1-8476-bfac922bc884 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:46:59.777803 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:46:59.781748 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:46:59.782278 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:46:59.933987 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Pushing event updated for resources: {'Port': ['ID=a664371b-0c78-4e7b-9b16-cdeee995b663,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:46:59.945538 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.241s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:00.053247 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-83ab2b27-740f-4534-9d49-67f9f87ee025 None None] Provisioning for port a664371b-0c78-4e7b-9b16-cdeee995b663 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:47:00.058186 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-83ab2b27-740f-4534-9d49-67f9f87ee025 None None] Provisioning complete for port a664371b-0c78-4e7b-9b16-cdeee995b663 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:47:00.058563 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-83ab2b27-740f-4534-9d49-67f9f87ee025 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:00.115005 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-83ab2b27-740f-4534-9d49-67f9f87ee025 None None] Port a664371b-0c78-4e7b-9b16-cdeee995b663 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:47:00.166245 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:00.411723 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:00.481676 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:00.482795 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:00.486868 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:00.624236 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Pushing event updated for resources: {'Port': ['ID=a664371b-0c78-4e7b-9b16-cdeee995b663,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:00.628938 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.142s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:00.690620 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Network e758f9f9-1a52-43f1-8476-bfac922bc884 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:47:00.691670 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:47:00.939344 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-1f5b9165-a2e5-4e44-9c38-536b817f1a56 None None] Provisioning complete for port a664371b-0c78-4e7b-9b16-cdeee995b663 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:47:00.939777 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1f5b9165-a2e5-4e44-9c38-536b817f1a56 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:00.962301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Router b06623f8-649a-45fb-9b57-d4616c706968 is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99040) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 11:47:00.994304 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-1f5b9165-a2e5-4e44-9c38-536b817f1a56 None None] Port a664371b-0c78-4e7b-9b16-cdeee995b663 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:47:01.120952 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:47:01.130623 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Sync routers for ids ['b06623f8-649a-45fb-9b57-d4616c706968'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:47:01.131699 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] 10.222.0.48 "PUT /v2.0/routers/b06623f8-649a-45fb-9b57-d4616c706968/add_router_interface HTTP/1.1" status: 200 len: 508 time: 2.4452302 May 31 11:47:01.141192 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Routers {'b06623f8-649a-45fb-9b57-d4616c706968'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:47:01.142110 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:46:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['b06623f8-649a-45fb-9b57-d4616c706968'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:47:01.324906 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Set router b06623f8-649a-45fb-9b57-d4616c706968 gateway port host: None {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:47:01.721019 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'name': 'router1', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:46:49Z', 'updated_at': '2021-05-31T11:47:00Z', 'revision_number': 2, 'project_id': '8be0fb13457d403886185c544e983ded', 'gw_port_host': None, '_interfaces': [{'id': 'a664371b-0c78-4e7b-9b16-cdeee995b663', 'name': '', 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'mac_address': 'fa:16:3e:0c:be:a5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:46:59Z', 'updated_at': '2021-05-31T11:47:00Z', 'revision_number': 2, 'project_id': '8be0fb13457d403886185c544e983ded', 'subnets': [{'id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'a90ae401-2f98-4796-953a-0a02d0edcb63'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:47:01.725962 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Checking router: b06623f8-649a-45fb-9b57-d4616c706968 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:47:01.726335 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Updating router b06623f8-649a-45fb-9b57-d4616c706968 port a664371b-0c78-4e7b-9b16-cdeee995b663 binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:47:01.777962 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:02.185691 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:02.203470 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Transition to ACTIVE for port object a664371b-0c78-4e7b-9b16-cdeee995b663 will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:47:02.274415 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:02.279658 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:02.429982 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Pushing event updated for resources: {'Port': ['ID=a664371b-0c78-4e7b-9b16-cdeee995b663,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:02.481867 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.482436 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.482978 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.483468 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.483943 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.484955 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.485472 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.486051 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.486630 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.487188 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.487744 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.488304 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Tracked resource versions report: May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: pushed versions: May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: { 'AddressGroup': {'1.2'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'ConntrackHelper': {'1.0'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Log': {'1.0'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Network': {'1.1'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Port': {'1.7'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'PortForwarding': {'1.2'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'QosPolicy': {'1.8'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SecurityGroup': {'1.4'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SecurityGroupRule': {'1.2'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SubPort': {'1.0'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Subnet': {'1.1'}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Trunk': {'1.1'}} May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: consumer versions: May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: { DHCP agent@dv-cinder-791239-4-iscsi: {}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: L3 agent@dv-cinder-791239-4-iscsi: {}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: Metadata agent@dv-cinder-791239-4-iscsi: {}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: Metering agent@dv-cinder-791239-4-iscsi: {}, May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: Open vSwitch agent@dv-cinder-791239-4-iscsi: { 'AddressGroup': '1.2', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'ConntrackHelper': '1.0', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Log': '1.0', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Network': '1.1', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Port': '1.7', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'PortForwarding': '1.2', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'QosPolicy': '1.8', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SecurityGroup': '1.4', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SecurityGroupRule': '1.2', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SubPort': '1.0', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Subnet': '1.1', May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Trunk': '1.1'}} May 31 11:47:02.490254 dv-cinder-791239-4-iscsi neutron-server[99042]: {{(pid=99042) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 31 11:47:02.500656 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.220s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:02.566963 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Attempting to bind port a664371b-0c78-4e7b-9b16-cdeee995b663 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:47:02.567388 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Attempting to bind port a664371b-0c78-4e7b-9b16-cdeee995b663 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:47:02.567816 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Attempting to bind port a664371b-0c78-4e7b-9b16-cdeee995b663 on network e758f9f9-1a52-43f1-8476-bfac922bc884 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:47:02.574008 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 46, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:47:02.575174 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Checking segment: {'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:47:02.575174 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:02.575371 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Bound using segment: {'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:47:02.575758 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Bound port: a664371b-0c78-4e7b-9b16-cdeee995b663, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 384, 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:47:02.576126 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:02.675391 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] For port a664371b-0c78-4e7b-9b16-cdeee995b663, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:47:02.679353 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] For port a664371b-0c78-4e7b-9b16-cdeee995b663, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=a664371b-0c78-4e7b-9b16-cdeee995b663,segment=NetworkSegment(ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee),segment_id=ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:47:02.772047 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:02.776192 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:02.876614 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Pushing event updated for resources: {'Port': ['ID=a664371b-0c78-4e7b-9b16-cdeee995b663,revision_number=4']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:02.880839 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:02.886219 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Routers {'b06623f8-649a-45fb-9b57-d4616c706968'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:47:02.886855 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:46:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['b06623f8-649a-45fb-9b57-d4616c706968'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:47:03.101178 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] Set router b06623f8-649a-45fb-9b57-d4616c706968 gateway port host: None {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:47:03.420007 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 55808) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:47:03.450310 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'name': 'router1', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:46:49Z', 'updated_at': '2021-05-31T11:47:00Z', 'revision_number': 2, 'project_id': '8be0fb13457d403886185c544e983ded', 'gw_port_host': None, '_interfaces': [{'id': 'a664371b-0c78-4e7b-9b16-cdeee995b663', 'name': '', 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'mac_address': 'fa:16:3e:0c:be:a5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:46:59Z', 'updated_at': '2021-05-31T11:47:02Z', 'revision_number': 4, 'project_id': '8be0fb13457d403886185c544e983ded', 'subnets': [{'id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'a90ae401-2f98-4796-953a-0a02d0edcb63'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:47:03.454666 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b1b4b086-f4a3-483c-b1b5-02c917e64c2a None None] The sync data for ids ['b06623f8-649a-45fb-9b57-d4616c706968'] in dv-cinder-791239-4-iscsi is: [{'id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'name': 'router1', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:46:49Z', 'updated_at': '2021-05-31T11:47:00Z', 'revision_number': 2, 'project_id': '8be0fb13457d403886185c544e983ded', 'gw_port_host': None, '_interfaces': [{'id': 'a664371b-0c78-4e7b-9b16-cdeee995b663', 'name': '', 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'mac_address': 'fa:16:3e:0c:be:a5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:46:59Z', 'updated_at': '2021-05-31T11:47:02Z', 'revision_number': 4, 'project_id': '8be0fb13457d403886185c544e983ded', 'subnets': [{'id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'a90ae401-2f98-4796-953a-0a02d0edcb63'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:47:03.609847 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c3881d29-1514-443b-9e0f-aaa95a81b6c2 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:47:03.611239 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c3881d29-1514-443b-9e0f-aaa95a81b6c2 admin admin] 10.222.0.48 "GET /v2.0/networks/8a73d753-6759-4d77-a99f-016e97b0a680 HTTP/1.1" status: 200 len: 842 time: 0.1901259 May 31 11:47:03.625133 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Request body: {'subnet': {'enable_dhcp': False, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'gateway_ip': '172.27.1.1', 'ip_version': 4, 'cidr': '172.27.1.0/24', 'name': 'public-subnet'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:47:03.630238 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:47:03.634003 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Usage tracker for resource:subnet and tenant:cf8b125186c446b2aec91a3f69c5f1fa is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:47:03.636943 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:47:03.643049 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:47:03.836507 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Made reservation on behalf of cf8b125186c446b2aec91a3f69c5f1fa for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:47:03.839182 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Notify callbacks [] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:04.377630 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.objects.ports [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Time-cost: Fetching 0 ports in 0 security groups {{(pid=99039) get_objects /opt/stack/neutron/neutron/objects/ports.py:443}} May 31 11:47:04.378108 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:04.383690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:04.463181 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:47:04.463823 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Network hostable DHCP agents. Network: 8a73d753-6759-4d77-a99f-016e97b0a680, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:47:04.467247 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Pushing event updated for resources: {'Subnet': ['ID=4b97143c-47ad-4e8e-b97f-9970a81c7a23,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:04.477966 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.478305 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.478634 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.478950 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.479265 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.479588 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.479900 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.480209 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.480534 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.480847 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.481179 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.481485 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Tracked resource versions report: May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: pushed versions: May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: { 'AddressGroup': {'1.2'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'ConntrackHelper': {'1.0'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Log': {'1.0'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Network': {'1.1'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Port': {'1.7'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'PortForwarding': {'1.2'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'QosPolicy': {'1.8'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SecurityGroup': {'1.4'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SecurityGroupRule': {'1.2'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SubPort': {'1.0'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Subnet': {'1.1'}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Trunk': {'1.1'}} May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: consumer versions: May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: { DHCP agent@dv-cinder-791239-4-iscsi: {}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: L3 agent@dv-cinder-791239-4-iscsi: {}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: Metadata agent@dv-cinder-791239-4-iscsi: {}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: Metering agent@dv-cinder-791239-4-iscsi: {}, May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: Open vSwitch agent@dv-cinder-791239-4-iscsi: { 'AddressGroup': '1.2', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'ConntrackHelper': '1.0', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Log': '1.0', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Network': '1.1', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Port': '1.7', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'PortForwarding': '1.2', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'QosPolicy': '1.8', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SecurityGroup': '1.4', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SecurityGroupRule': '1.2', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SubPort': '1.0', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Subnet': '1.1', May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Trunk': '1.1'}} May 31 11:47:04.482549 dv-cinder-791239-4-iscsi neutron-server[99039]: {{(pid=99039) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 31 11:47:04.534840 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.151s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:04.624822 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:47:04.721848 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 8a73d753-6759-4d77-a99f-016e97b0a680) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:47:04.726529 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:47:04.727140 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:04.732869 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-509c02c5-3c7e-404b-8355-6f133cf19c92 None None] Network 8a73d753-6759-4d77-a99f-016e97b0a680 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:47:04.837746 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 821 time: 1.2160816 May 31 11:47:04.944895 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e657bc68-e7f3-4f2b-936d-3dc79af657af None None] Network 8a73d753-6759-4d77-a99f-016e97b0a680 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:47:07.196427 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 55826) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:47:07.372244 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device a664371b-0c78-4e7b-9b16-cdeee995b663 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:47:07.408890 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1f4bdf23-459b-44f4-9f57-eb2fc02a8e8f admin admin] 10.222.0.48 "GET /v2.0/routers/b06623f8-649a-45fb-9b57-d4616c706968 HTTP/1.1" status: 200 len: 710 time: 0.2110951 May 31 11:47:07.527403 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port a664371b-0c78-4e7b-9b16-cdeee995b663 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:47:07.532132 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port a664371b-0c78-4e7b-9b16-cdeee995b663 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:47:07.532574 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:07.537485 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-18852d50-b7e1-4cc9-8c65-cd997e932e45 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:47:07.543294 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-18852d50-b7e1-4cc9-8c65-cd997e932e45 admin admin] 10.222.0.48 "GET /v2.0/networks/8a73d753-6759-4d77-a99f-016e97b0a680 HTTP/1.1" status: 200 len: 886 time: 0.1246450 May 31 11:47:07.548188 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Request body: {'router': {'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:47:07.650674 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:07.682247 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port a664371b-0c78-4e7b-9b16-cdeee995b663, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=a664371b-0c78-4e7b-9b16-cdeee995b663,segment=NetworkSegment(ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee),segment_id=ca66c0f5-0ff8-43e1-afd7-bc0a1e60dbee)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:47:07.812879 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:07.812879 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:07.877725 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=a664371b-0c78-4e7b-9b16-cdeee995b663,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:07.881472 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:07.991299 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:08.411080 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:08.465409 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:08.631398 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Allocated IP 172.27.1.71 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/062938e6-5834-4fe6-8074-5e81a1edb8d7) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:47:08.775835 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:08.920474 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:08.924088 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:08.982535 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:47:08.985525 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:47:08.989226 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:47:09.053194 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Pushing event updated for resources: {'Port': ['ID=062938e6-5834-4fe6-8074-5e81a1edb8d7,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:09.103314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:09.337325 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device a664371b-0c78-4e7b-9b16-cdeee995b663 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:47:09.404295 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port a664371b-0c78-4e7b-9b16-cdeee995b663 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:47:09.404668 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:09.482966 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:09.848735 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:09.851909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_distributed_port_bindings_after_change-507836', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_snat_interfaces_after_change-507901', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._reconfigure_ha_resources-505705'] for router, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:10.086286 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Router b06623f8-649a-45fb-9b57-d4616c706968 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:47:10.265721 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:47:10.274918 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:10.276080 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] 10.222.0.48 "PUT /v2.0/routers/b06623f8-649a-45fb-9b57-d4616c706968 HTTP/1.1" status: 200 len: 889 time: 2.7316904 May 31 11:47:10.276829 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Sync routers for ids ['b06623f8-649a-45fb-9b57-d4616c706968'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:47:10.287479 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Routers {'b06623f8-649a-45fb-9b57-d4616c706968'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:47:10.288114 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:46:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['b06623f8-649a-45fb-9b57-d4616c706968'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:47:10.519088 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Set router b06623f8-649a-45fb-9b57-d4616c706968 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:47:10.878138 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'name': 'router1', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.71'}], 'enable_snat': True}, 'gw_port_id': '062938e6-5834-4fe6-8074-5e81a1edb8d7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:46:49Z', 'updated_at': '2021-05-31T11:47:09Z', 'revision_number': 4, 'project_id': '8be0fb13457d403886185c544e983ded', 'gw_port': {'id': '062938e6-5834-4fe6-8074-5e81a1edb8d7', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c5:b9:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:47:08Z', 'updated_at': '2021-05-31T11:47:08Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'a664371b-0c78-4e7b-9b16-cdeee995b663', 'name': '', 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'mac_address': 'fa:16:3e:0c:be:a5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:46:59Z', 'updated_at': '2021-05-31T11:47:07Z', 'revision_number': 5, 'project_id': '8be0fb13457d403886185c544e983ded', 'subnets': [{'id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'a90ae401-2f98-4796-953a-0a02d0edcb63'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:47:10.889070 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Checking router: b06623f8-649a-45fb-9b57-d4616c706968 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:47:10.889442 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Updating router b06623f8-649a-45fb-9b57-d4616c706968 port 062938e6-5834-4fe6-8074-5e81a1edb8d7 binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:47:10.946221 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:11.299053 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:11.309929 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Transition to ACTIVE for port object 062938e6-5834-4fe6-8074-5e81a1edb8d7 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:47:11.378794 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:11.382743 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:11.458788 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Attempting to bind port 062938e6-5834-4fe6-8074-5e81a1edb8d7 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:47:11.459151 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Attempting to bind port 062938e6-5834-4fe6-8074-5e81a1edb8d7 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:47:11.459509 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Attempting to bind port 062938e6-5834-4fe6-8074-5e81a1edb8d7 on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:47:11.466398 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4a8b2014-a16c-45d0-8004-336760731c2a None None] Pushing event updated for resources: {'Port': ['ID=062938e6-5834-4fe6-8074-5e81a1edb8d7,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:11.469171 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 46, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:47:11.469542 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:47:11.469964 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:11.470341 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:47:11.470809 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Bound port: 062938e6-5834-4fe6-8074-5e81a1edb8d7, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:47:11.471176 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:11.494594 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-4a8b2014-a16c-45d0-8004-336760731c2a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:11.543218 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] For port 062938e6-5834-4fe6-8074-5e81a1edb8d7, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:47:11.546050 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] For port 062938e6-5834-4fe6-8074-5e81a1edb8d7, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=062938e6-5834-4fe6-8074-5e81a1edb8d7,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:47:11.643521 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:11.647508 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:11.726798 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Routers {'b06623f8-649a-45fb-9b57-d4616c706968'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:47:11.727407 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:46:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['b06623f8-649a-45fb-9b57-d4616c706968'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:47:11.778610 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4a8b2014-a16c-45d0-8004-336760731c2a None None] Pushing event updated for resources: {'Port': ['ID=062938e6-5834-4fe6-8074-5e81a1edb8d7,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:11.788654 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-4a8b2014-a16c-45d0-8004-336760731c2a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.141s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:11.911618 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] Set router b06623f8-649a-45fb-9b57-d4616c706968 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:47:12.281630 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'name': 'router1', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.71'}], 'enable_snat': True}, 'gw_port_id': '062938e6-5834-4fe6-8074-5e81a1edb8d7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:46:49Z', 'updated_at': '2021-05-31T11:47:09Z', 'revision_number': 4, 'project_id': '8be0fb13457d403886185c544e983ded', 'gw_port': {'id': '062938e6-5834-4fe6-8074-5e81a1edb8d7', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c5:b9:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:47:08Z', 'updated_at': '2021-05-31T11:47:11Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'a664371b-0c78-4e7b-9b16-cdeee995b663', 'name': '', 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'mac_address': 'fa:16:3e:0c:be:a5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:46:59Z', 'updated_at': '2021-05-31T11:47:07Z', 'revision_number': 5, 'project_id': '8be0fb13457d403886185c544e983ded', 'subnets': [{'id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'a90ae401-2f98-4796-953a-0a02d0edcb63'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:47:12.285870 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b3312aca-2441-4100-8100-31ccd5df9101 None None] The sync data for ids ['b06623f8-649a-45fb-9b57-d4616c706968'] in dv-cinder-791239-4-iscsi is: [{'id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'name': 'router1', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.71'}], 'enable_snat': True}, 'gw_port_id': '062938e6-5834-4fe6-8074-5e81a1edb8d7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:46:49Z', 'updated_at': '2021-05-31T11:47:09Z', 'revision_number': 4, 'project_id': '8be0fb13457d403886185c544e983ded', 'gw_port': {'id': '062938e6-5834-4fe6-8074-5e81a1edb8d7', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c5:b9:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:47:08Z', 'updated_at': '2021-05-31T11:47:11Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'a664371b-0c78-4e7b-9b16-cdeee995b663', 'name': '', 'network_id': 'e758f9f9-1a52-43f1-8476-bfac922bc884', 'tenant_id': '8be0fb13457d403886185c544e983ded', 'mac_address': 'fa:16:3e:0c:be:a5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b06623f8-649a-45fb-9b57-d4616c706968', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:46:59Z', 'updated_at': '2021-05-31T11:47:07Z', 'revision_number': 5, 'project_id': '8be0fb13457d403886185c544e983ded', 'subnets': [{'id': '5ba76f57-f24a-4beb-8571-668a63f94cf9', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'a90ae401-2f98-4796-953a-0a02d0edcb63'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:47:12.678384 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 55836) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:47:12.798716 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-cacb7cef-26aa-4514-ab8f-fd8f08b2f38e admin admin] 10.222.0.48 "GET /v2.0/ports?device_owner=network%3Arouter_gateway&fields=id&fields=name&fields=mac_address&fields=fixed_ips&fields=status HTTP/1.1" status: 200 len: 410 time: 0.1191986 May 31 11:47:13.366115 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 062938e6-5834-4fe6-8074-5e81a1edb8d7 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:47:13.481818 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 062938e6-5834-4fe6-8074-5e81a1edb8d7 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:47:13.487135 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 062938e6-5834-4fe6-8074-5e81a1edb8d7 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:47:13.487506 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:13.605508 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:13.625990 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 062938e6-5834-4fe6-8074-5e81a1edb8d7, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=062938e6-5834-4fe6-8074-5e81a1edb8d7,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:47:13.714341 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:13.716435 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-4a8b2014-a16c-45d0-8004-336760731c2a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:47:13.783469 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=062938e6-5834-4fe6-8074-5e81a1edb8d7,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:47:13.786582 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:47:15.369321 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 062938e6-5834-4fe6-8074-5e81a1edb8d7 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:47:15.440251 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 062938e6-5834-4fe6-8074-5e81a1edb8d7 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:47:15.440627 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:15.459147 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:47:20.914728 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f13a9db3-bbdd-4721-87b2-fbd4bde345c7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:23.361695 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:23.500931 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-4b5dae36-2377-4a18-a649-a99f94749e5f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:24.515911 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:26.024426 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:50.856181 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-838dc892-1ac4-49d0-a9c4-8a7041fb0a74 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:52.458616 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:47:53.321220 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:53.501678 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-4acb9218-e926-4c05-9cd1-baca41bee4c5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:55.578675 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:47:56.035596 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:20.874792 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3b1e21bf-0772-4fc1-9a4b-c2b9d9955c8a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:23.323103 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:23.497965 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-cce8ea04-2877-4090-8eb4-75ee3dc98f37 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:24.554564 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:26.080835 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:29.457386 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:48:50.219441 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56406) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:48:50.287333 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.pecan_wsgi.hooks.translation [None req-f4b7b664-91c9-41c1-bbcd-1a8fb4ffc76c admin admin] GET failed (client error): The resource could not be found. May 31 11:48:50.288567 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f4b7b664-91c9-41c1-bbcd-1a8fb4ffc76c admin admin] 10.222.0.48 "GET /v2.0/networks/private HTTP/1.1" status: 404 len: 315 time: 0.0682011 May 31 11:48:50.419146 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b59e5405-d1b7-4738-8f54-2775ae2ce362 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:48:50.420497 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-b59e5405-d1b7-4738-8f54-2775ae2ce362 admin admin] 10.222.0.48 "GET /v2.0/networks?name=private HTTP/1.1" status: 200 len: 872 time: 0.1281817 May 31 11:48:50.870595 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-41968c46-d106-4c74-b183-cbecf0d85e54 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:52.543300 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56416) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:48:52.651998 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6ea26c35-82ca-41b4-ba5f-a5846c1e9cc7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:48:52.652547 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6ea26c35-82ca-41b4-ba5f-a5846c1e9cc7 admin admin] Synchronizing usage tracker for tenant:cf8b125186c446b2aec91a3f69c5f1fa on resource:subnet {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:48:52.768014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6ea26c35-82ca-41b4-ba5f-a5846c1e9cc7 admin admin] Unset dirty status for tenant:cf8b125186c446b2aec91a3f69c5f1fa on resource:subnet {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:48:52.769097 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-6ea26c35-82ca-41b4-ba5f-a5846c1e9cc7 admin admin] 10.222.0.48 "GET /v2.0/subnets/5ba76f57-f24a-4beb-8571-668a63f94cf9 HTTP/1.1" status: 200 len: 849 time: 0.2248545 May 31 11:48:52.778399 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Request body: {'subnet': {'dns_nameservers': ['8.8.8.8', '8.8.4.4']}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:48:52.894248 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Notify callbacks [] for subnet, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:53.150811 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_subnet_gateway_ip_update-8759697376413'] for subnet, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:53.155062 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e2e3974a-ebbb-48ad-bbf0-6d0ec42e725c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:48:53.196076 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:48:53.196950 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:53.197922 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] 10.222.0.48 "PUT /v2.0/subnets/5ba76f57-f24a-4beb-8571-668a63f94cf9 HTTP/1.1" status: 200 len: 868 time: 0.4233668 May 31 11:48:53.220807 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-32594e46-320a-4382-b873-e98ff10add22 None None] Network e758f9f9-1a52-43f1-8476-bfac922bc884 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:48:53.222477 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Pushing event updated for resources: {'Subnet': ['ID=5ba76f57-f24a-4beb-8571-668a63f94cf9,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:48:53.230953 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.231332 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.231654 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.231974 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.232293 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.232626 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.232942 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.233252 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.233577 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.234072 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.234194 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.234643 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:48:53.238821 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:48:53.395863 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:53.507200 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-7344e6e2-760d-4061-aefa-8cbafcd1ce36 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:53.689407 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-2cd8ddc6-81f8-4dd6-9a8d-01f006fbb7b2 None None] Provisioning complete for port a664371b-0c78-4e7b-9b16-cdeee995b663 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:48:53.689802 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2cd8ddc6-81f8-4dd6-9a8d-01f006fbb7b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:53.827357 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-2cd8ddc6-81f8-4dd6-9a8d-01f006fbb7b2 None None] Provisioning complete for port 8eb0f362-8d0c-45b4-aaa8-4601b4e7de24 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:48:53.827614 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2cd8ddc6-81f8-4dd6-9a8d-01f006fbb7b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:54.533706 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:55.311426 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56426) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:48:55.487028 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6def540d-7ece-4881-97d6-0da37dc2869b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:48:55.487498 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6def540d-7ece-4881-97d6-0da37dc2869b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:48:55.488994 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-6def540d-7ece-4881-97d6-0da37dc2869b admin admin] 10.222.0.48 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1549 time: 0.1766238 May 31 11:48:56.051537 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:48:57.603851 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56436) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:48:57.774881 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-dd72f229-4d91-4be3-bfb3-697d94e118a9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:48:57.775376 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-dd72f229-4d91-4be3-bfb3-697d94e118a9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:48:57.776074 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-dd72f229-4d91-4be3-bfb3-697d94e118a9 admin admin] Synchronizing usage tracker for tenant:cf8b125186c446b2aec91a3f69c5f1fa on resource:network {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:48:57.851178 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-dd72f229-4d91-4be3-bfb3-697d94e118a9 admin admin] Unset dirty status for tenant:cf8b125186c446b2aec91a3f69c5f1fa on resource:network {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:48:57.852300 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-dd72f229-4d91-4be3-bfb3-697d94e118a9 admin admin] 10.222.0.48 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1549 time: 0.2474737 May 31 11:48:59.969660 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56446) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:00.136912 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-8c1ab17a-9f64-424c-9ef0-f14e8ecbf6f3 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:00.138521 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-8c1ab17a-9f64-424c-9ef0-f14e8ecbf6f3 admin admin] 10.222.0.48 "GET /v2.0/networks/8a73d753-6759-4d77-a99f-016e97b0a680 HTTP/1.1" status: 200 len: 886 time: 0.1677654 May 31 11:49:02.280139 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56456) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:02.454531 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0a82ca95-aaf0-4b82-8a24-fb37b0cdb333 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:02.455834 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-0a82ca95-aaf0-4b82-8a24-fb37b0cdb333 admin admin] 10.222.0.48 "GET /v2.0/networks/e758f9f9-1a52-43f1-8476-bfac922bc884 HTTP/1.1" status: 200 len: 869 time: 0.1747737 May 31 11:49:04.583223 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56466) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:04.692221 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ad036429-1a08-4ca1-909b-50d3fed25acd admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:04.692681 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ad036429-1a08-4ca1-909b-50d3fed25acd admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:04.694186 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-ad036429-1a08-4ca1-909b-50d3fed25acd admin admin] 10.222.0.48 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1099482 May 31 11:49:06.456464 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:49:06.802500 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56476) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:06.912479 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-fcda7439-3ea1-4824-abf3-0f021ec776f9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:06.912942 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-fcda7439-3ea1-4824-abf3-0f021ec776f9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:06.914477 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-fcda7439-3ea1-4824-abf3-0f021ec776f9 admin admin] 10.222.0.48 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1110179 May 31 11:49:09.045034 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56486) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:09.146030 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e5df6d49-7220-49e8-be45-1d0f52ee589b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:09.147323 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-e5df6d49-7220-49e8-be45-1d0f52ee589b admin admin] 10.222.0.48 "GET /v2.0/subnets/4b97143c-47ad-4e8e-b97f-9970a81c7a23 HTTP/1.1" status: 200 len: 816 time: 0.1013343 May 31 11:49:11.266890 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56496) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:11.377172 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b8a18a23-347d-4244-b404-5dc259dfb7b5 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:11.378488 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-b8a18a23-347d-4244-b404-5dc259dfb7b5 admin admin] 10.222.0.48 "GET /v2.0/subnets/5ba76f57-f24a-4beb-8571-668a63f94cf9 HTTP/1.1" status: 200 len: 868 time: 0.1106861 May 31 11:49:20.869443 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-fca3476b-d510-4d2d-a288-ad2410138039 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:23.315014 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:23.473516 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-149d8b8b-47b2-4355-acf7-a953667bd3f1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:24.521005 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:26.029393 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:34.670315 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56540) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:34.716699 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [None req-67878271-f782-4cc8-89ec-edbfb61f53dc admin admin] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99039) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:49:34.717729 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-67878271-f782-4cc8-89ec-edbfb61f53dc admin admin] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0464473 May 31 11:49:36.933548 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56550) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:36.998851 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.pecan_wsgi.hooks.translation [None req-aac28fa2-8b3f-4a20-ac3c-f86c91a5aa70 admin admin] GET failed (client error): The resource could not be found. May 31 11:49:37.000088 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-aac28fa2-8b3f-4a20-ac3c-f86c91a5aa70 admin admin] 10.222.0.48 "GET /v2.0/networks/public HTTP/1.1" status: 404 len: 314 time: 0.0656030 May 31 11:49:37.124686 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f6b2a2f5-cb11-4d6c-a36f-11adcc1c98e2 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:37.126063 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f6b2a2f5-cb11-4d6c-a36f-11adcc1c98e2 admin admin] 10.222.0.48 "GET /v2.0/networks?name=public HTTP/1.1" status: 200 len: 889 time: 0.1220903 May 31 11:49:39.271976 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56560) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:39.321016 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Request body: {'network': {'name': 'shared', 'admin_state_up': True, 'shared': True}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:49:39.326458 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:49:39.336070 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Attempting to reserve 1 items for resource network. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:49:39.399963 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Made reservation on behalf of cf8b125186c446b2aec91a3f69c5f1fa for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:49:39.402776 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:39.535806 dv-cinder-791239-4-iscsi neutron-server[99039]: WARNING neutron.services.revisions.revision_plugin [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Could not find related network for resource to bump revision. May 31 11:49:39.549965 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] vlan segment allocate from pool success with {'vlan_id': 358, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:49:39.560320 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8759697518309', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:39.560865 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Unsubscribe: segment precommit_create {{(pid=99039) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 31 11:49:39.561321 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Added segment a7256a82-57c5-4819-8aea-511768fbafdc of type vlan for network 1b392bba-a31f-4f82-a1fa-40181dc52c6b May 31 11:49:39.597216 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:39.680167 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:39.681494 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:39.682124 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:39.686496 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:49:39.782320 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Pushing event updated for resources: {'Network': ['ID=1b392bba-a31f-4f82-a1fa-40181dc52c6b,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:49:39.785477 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:49:39.792471 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 1b392bba-a31f-4f82-a1fa-40181dc52c6b details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:49:39.834999 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 0.5613036 May 31 11:49:41.957121 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56570) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:49:42.023148 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.pecan_wsgi.hooks.translation [None req-fc13f269-41eb-49d0-b9ae-aeb8569defc3 admin admin] GET failed (client error): The resource could not be found. May 31 11:49:42.023642 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-fc13f269-41eb-49d0-b9ae-aeb8569defc3 admin admin] Synchronizing usage tracker for tenant:cf8b125186c446b2aec91a3f69c5f1fa on resource:network {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:49:42.125946 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-fc13f269-41eb-49d0-b9ae-aeb8569defc3 admin admin] Unset dirty status for tenant:cf8b125186c446b2aec91a3f69c5f1fa on resource:network {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:49:42.127094 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-fc13f269-41eb-49d0-b9ae-aeb8569defc3 admin admin] 10.222.0.48 "GET /v2.0/networks/shared HTTP/1.1" status: 404 len: 314 time: 0.1689672 May 31 11:49:42.232635 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-610a555a-6c14-4dfe-b680-04479ab60790 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:42.234149 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-610a555a-6c14-4dfe-b680-04479ab60790 admin admin] 10.222.0.48 "GET /v2.0/networks?name=shared HTTP/1.1" status: 200 len: 826 time: 0.1031532 May 31 11:49:42.243679 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Request body: {'subnet': {'description': 'shared-subnet', 'name': 'shared-subnet', 'cidr': '192.168.233.0/24', 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b', 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:49:42.248671 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:49:42.258142 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:49:42.314488 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Made reservation on behalf of cf8b125186c446b2aec91a3f69c5f1fa for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:49:42.316709 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:42.741841 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:42.746713 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:49:42.818696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Pushing event updated for resources: {'Subnet': ['ID=0baa2a0d-0223-4ead-bf59-0b788515e70d,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:49:42.823669 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:49:42.826107 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Checking segment: {'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:49:42.826456 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Network hostable DHCP agents. Network: 1b392bba-a31f-4f82-a1fa-40181dc52c6b, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:49:42.914570 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Network 1b392bba-a31f-4f82-a1fa-40181dc52c6b is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:49:43.039742 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 1b392bba-a31f-4f82-a1fa-40181dc52c6b) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:49:43.044385 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:49:43.045042 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:43.048986 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-efc75165-69ce-44b9-a1ad-2ae9fe0ddbff None None] Network 1b392bba-a31f-4f82-a1fa-40181dc52c6b requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:49:43.149742 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.9093127 May 31 11:49:43.288326 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-1b392bba-a31f-4f82-a1fa-40181dc52c6b', 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b', 'tenant_id': 'cf8b125186c446b2aec91a3f69c5f1fa', 'fixed_ips': [{'subnet_id': '0baa2a0d-0223-4ead-bf59-0b788515e70d'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 11:49:43.289170 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:43.447613 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Allocated IP 192.168.233.2 (1b392bba-a31f-4f82-a1fa-40181dc52c6b/0baa2a0d-0223-4ead-bf59-0b788515e70d/07f88329-44aa-4a2b-a1b8-2c6c45faa4a3) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:49:43.454910 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:49:43.660708 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:43.667065 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Transition to ACTIVE for port object 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:49:43.686230 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Transition to ACTIVE for port object 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:49:43.753901 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:43.758503 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:49:43.824871 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Network 1b392bba-a31f-4f82-a1fa-40181dc52c6b is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:49:43.827460 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:49:43.832008 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Pushing event updated for resources: {'Port': ['ID=07f88329-44aa-4a2b-a1b8-2c6c45faa4a3,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:49:43.836476 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:49:43.866595 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.866924 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.867258 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.867578 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.867891 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.868205 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.868513 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.868846 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.869175 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.869485 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.869921 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.870215 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:49:43.878612 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:49:43.936295 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Attempting to bind port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:49:43.936574 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Attempting to bind port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:49:43.936927 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Attempting to bind port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 on network 1b392bba-a31f-4f82-a1fa-40181dc52c6b {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:49:43.942141 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 49, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:49:43.942502 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Checking segment: {'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:49:43.942892 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:43.943273 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Bound using segment: {'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:49:43.943728 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Bound port: 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:49:43.944088 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:44.031018 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] For port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:49:44.034255 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] For port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=07f88329-44aa-4a2b-a1b8-2c6c45faa4a3,segment=NetworkSegment(a7256a82-57c5-4819-8aea-511768fbafdc),segment_id=a7256a82-57c5-4819-8aea-511768fbafdc)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:49:44.132295 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:44.136969 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:49:44.214087 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Pushing event updated for resources: {'Port': ['ID=07f88329-44aa-4a2b-a1b8-2c6c45faa4a3,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:49:44.217239 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:49:44.999989 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-93ad2405-04a3-417c-8d95-cb6740a76ead None None] Network 1b392bba-a31f-4f82-a1fa-40181dc52c6b requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:49:45.152144 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-7d569707-1491-429c-8112-f6fb7199d79c None None] Provisioning for port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:49:45.486700 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:49:45.617145 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:49:45.622837 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:49:45.623202 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:45.744438 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:45.776719 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=07f88329-44aa-4a2b-a1b8-2c6c45faa4a3,segment=NetworkSegment(a7256a82-57c5-4819-8aea-511768fbafdc),segment_id=a7256a82-57c5-4819-8aea-511768fbafdc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:49:45.834595 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:45.836752 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:49:45.902672 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=07f88329-44aa-4a2b-a1b8-2c6c45faa4a3,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:49:45.905684 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:49:47.463942 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:49:47.535849 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 07f88329-44aa-4a2b-a1b8-2c6c45faa4a3 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:49:47.536236 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:50.862738 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-d84d2402-a646-4f78-9e14-4d144ed31990 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:53.348077 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:53.474290 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-a6a840b7-457f-4734-b6e4-94fd610c643d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:54.548424 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:49:56.033936 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:20.453819 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:50:20.844379 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1fe8b220-b55f-40e6-bebe-825c0782fffb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:23.336322 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:23.507845 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e92196a5-cb72-4b1b-b899-8cd0bb96cb8f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:24.541938 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:26.044218 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:50.861393 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-58b5b347-adf7-46d4-9b20-5610f322cc76 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:53.312373 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:53.495713 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ef183083-b803-41be-8285-f83e07860b49 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:54.552198 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:56.021170 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:50:57.452031 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:51:02.571000 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56620) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:51:02.577003 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [None req-d6be42ef-3d61-4956-afc4-cb084baa97e7 tempest-verify_tempest_config-614916142 tempest-verify_tempest_config-614916142-project] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99039) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:51:02.578034 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d6be42ef-3d61-4956-afc4-cb084baa97e7 tempest-verify_tempest_config-614916142 tempest-verify_tempest_config-614916142-project] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13635 time: 0.0060222 May 31 11:51:03.060644 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56638) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:51:03.173350 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:03.187643 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:51:03.191371 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Usage tracker for resource:security_group_rule and tenant:032459be03aa4d339e137f55cc7f298e is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:51:03.194754 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:51:03.200218 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:51:03.287805 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:03.374938 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:03.382828 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:51:03.421114 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=032459be03aa4d339e137f55cc7f298e&name=default HTTP/1.1" status: 200 len: 2975 time: 0.3594677 May 31 11:51:03.422585 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2db52398-80bc-4638-ace1-9641297d89f2,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:51:03.423930 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56644) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:51:03.429534 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.429916 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.430247 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.430565 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.430879 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.431211 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.431522 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.431836 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.432159 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.432475 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.432779 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.433083 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.437302 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:51:03.484754 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:03.510165 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:03.660825 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:03.665242 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:51:03.676132 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2db52398-80bc-4638-ace1-9641297d89f2,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:51:03.686217 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.686574 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.686912 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.687239 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.687562 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.687898 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.688293 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.688585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.688917 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.689235 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.689548 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.689902 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:51:03.692920 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:03.701931 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/2db52398-80bc-4638-ace1-9641297d89f2 HTTP/1.1" status: 204 len: 168 time: 0.2718625 May 31 11:51:03.702881 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:51:20.906045 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1f02287c-bd93-4a14-a514-fca18bbe6e6e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:23.413133 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:23.511836 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ec77a8b1-b25b-4863-94dd-4dccf4cad135 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:24.535097 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:26.048859 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:34.451730 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:51:50.865128 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1572e097-4264-4099-a526-351e34c719f1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:53.311326 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:53.482806 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-266195ff-59cb-44f0-9bd3-83e60d4aec63 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:54.567271 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:51:56.035769 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:11.452832 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 5 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:52:20.858185 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-065dd9cb-d073-42dc-afee-6aff24d3ca00 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:23.310967 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:23.474544 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-bc91d605-dd53-47fa-846a-dfae8f1069bf None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:24.543288 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:26.051812 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:38.988789 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318'] for agent, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:42.677355 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56696) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:52:42.790120 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-57438745-7c04-4957-8f69-5c6b55df7767 admin admin] 10.222.0.48 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.1116490 May 31 11:52:44.899361 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56706) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:52:44.957215 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-e567ee2d-3b6b-40af-82db-3a60308207cb admin admin] 10.222.0.48 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.0567677 May 31 11:52:48.452277 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:52:50.851119 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-752e659a-e5a2-458e-954f-2766085e03ab None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:53.333218 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:53.511057 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dd3d56ed-9fa4-4cf7-8c9f-419aae531811 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:54.567802 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:52:56.085051 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:08.993537 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:20.929991 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dd28c290-2f17-4499-8955-170048573064 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:23.312805 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:23.507362 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-56e53807-b50a-4af3-8a1c-5a1cd559948a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:24.619081 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:25.451943 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:53:26.058920 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:38.946916 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:50.918680 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b826365-9a62-44fd-98c8-b88e8bdc5f60 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:53.346386 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:53.483242 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2bd1c09c-80a4-4e12-9c86-a971eea286f5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:54.566840 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:53:56.052845 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:02.450285 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:54:08.969166 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:12.837706 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56914) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:13.284670 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56930) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:13.330196 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56936) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:13.362970 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 56942) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:13.399691 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56948) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:13.430934 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeTestJSON-1240663657-network', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:13.436721 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.440533 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Usage tracker for resource:network and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.443378 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.449654 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.561545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Made reservation on behalf of b866e6eab1b04406a7280f91698983ce for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:13.563913 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.577383 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.597247 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Request body: {'network': {'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-network', 'tenant_id': 'c94978b38250481095d2824adfb16ed4'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:13.597881 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.603267 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Usage tracker for resource:network and tenant:c94978b38250481095d2824adfb16ed4 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.608618 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Request body: {'network': {'name': 'tempest-DeleteServersTestJSON-1571582306-network', 'tenant_id': '3b87c0540dea485d8ec60a8628fced11'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:13.611600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.625996 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeShelveTestJSON-831574982-network', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:13.630822 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.634474 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.637282 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.639797 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Usage tracker for resource:network and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.648005 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.656733 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeNegativeTest-1594415552-network', 'tenant_id': '7dea63b6c4a340d7b361fe91ca6b8281'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:13.658599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.662479 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Usage tracker for resource:security_group_rule and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.673177 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.677372 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.683497 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Usage tracker for resource:network and tenant:3b87c0540dea485d8ec60a8628fced11 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.686808 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.689376 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.697616 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.703262 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.708896 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Usage tracker for resource:network and tenant:7dea63b6c4a340d7b361fe91ca6b8281 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.752101 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Made reservation on behalf of 0e7f0a333ee54b31bcd5c2290c3e284d for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:13.754725 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.764022 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.771579 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.775003 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Made reservation on behalf of c94978b38250481095d2824adfb16ed4 for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:13.776402 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.785927 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.794202 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.796876 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.807134 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Usage tracker for resource:security_group_rule and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.807134 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.813621 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.820898 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.829314 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Usage tracker for resource:security_group_rule and tenant:c94978b38250481095d2824adfb16ed4 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.832795 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.884316 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.891540 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.893761 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Made reservation on behalf of 3b87c0540dea485d8ec60a8628fced11 for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:13.896185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.912871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.913049 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.933384 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:13.933384 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Usage tracker for resource:security_group_rule and tenant:3b87c0540dea485d8ec60a8628fced11 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:13.936145 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:13.942451 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:13.952155 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Made reservation on behalf of 7dea63b6c4a340d7b361fe91ca6b8281 for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:13.954711 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.973289 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.978459 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:13.982274 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bb9b88f5-9285-45e4-adce-25d595f5f78b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:13.998108 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.008667 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] vlan segment allocate from pool success with {'vlan_id': 363, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:54:14.018853 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.019184 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Added segment f84631f1-6292-4e5f-a5dd-68b963cdfc7f of type vlan for network 36cce22a-502c-464d-a9ac-50f2bc247e07 May 31 11:54:14.029685 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] vlan segment allocate from pool success with {'vlan_id': 371, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:54:14.040633 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f149b027-6e58-4aae-b1ae-31001ccb1da9,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:14.049302 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.054450 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.055318 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.055801 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.056669 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.057158 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.058245 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.058909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.059375 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.059850 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Tracked resource versions report: May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: pushed versions: May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: { 'AddressGroup': {'1.2'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'ConntrackHelper': {'1.0'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Log': {'1.0'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Network': {'1.1'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Port': {'1.7'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'PortForwarding': {'1.2'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'QosPolicy': {'1.8'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SecurityGroup': {'1.4'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SecurityGroupRule': {'1.2'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SubPort': {'1.0'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Subnet': {'1.1'}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Trunk': {'1.1'}} May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: consumer versions: May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: { DHCP agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: HyperV agent@hv-cinder-79123: {}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: L3 agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: Metadata agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: Metering agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: Open vSwitch agent@dv-cinder-791239-4-iscsi: { 'AddressGroup': '1.2', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'ConntrackHelper': '1.0', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Log': '1.0', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Network': '1.1', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Port': '1.7', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'PortForwarding': '1.2', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'QosPolicy': '1.8', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SecurityGroup': '1.4', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SecurityGroupRule': '1.2', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'SubPort': '1.0', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Subnet': '1.1', May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: 'Trunk': '1.1'}} May 31 11:54:14.064696 dv-cinder-791239-4-iscsi neutron-server[99039]: {{(pid=99039) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 31 11:54:14.068935 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.069620 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Added segment 15b69bd1-9fea-43fa-badd-39be1ac15eb5 of type vlan for network 19c09cdd-9420-4946-a105-024d5ea86ec3 May 31 11:54:14.071761 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:14.084728 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-88145257-27e8-44e4-b710-93e451b1a827 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:14.099060 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:14.108249 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.110141 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Usage tracker for resource:security_group_rule and tenant:7dea63b6c4a340d7b361fe91ca6b8281 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:14.114182 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:14.123523 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:14.175370 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.187458 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:14.188258 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.191807 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c242c4b8-d59a-410d-9edf-57e25eb35fd6,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:14.226025 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.226575 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.226902 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.227518 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.228000 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.228556 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.229041 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.229589 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.230079 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.230552 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.231024 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.231499 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Tracked resource versions report: May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: pushed versions: May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: { 'AddressGroup': {'1.2'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'ConntrackHelper': {'1.0'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Log': {'1.0'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Network': {'1.1'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Port': {'1.7'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'PortForwarding': {'1.2'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'QosPolicy': {'1.8'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SecurityGroup': {'1.4'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SecurityGroupRule': {'1.2'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SubPort': {'1.0'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Subnet': {'1.1'}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Trunk': {'1.1'}} May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: consumer versions: May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: { DHCP agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: HyperV agent@hv-cinder-79123: {}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: L3 agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: Metadata agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: Metering agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: Open vSwitch agent@dv-cinder-791239-4-iscsi: { 'AddressGroup': '1.2', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'ConntrackHelper': '1.0', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Log': '1.0', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Network': '1.1', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Port': '1.7', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'PortForwarding': '1.2', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'QosPolicy': '1.8', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SecurityGroup': '1.4', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SecurityGroupRule': '1.2', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'SubPort': '1.0', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Subnet': '1.1', May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: 'Trunk': '1.1'}} May 31 11:54:14.232658 dv-cinder-791239-4-iscsi neutron-server[99040]: {{(pid=99040) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 31 11:54:14.241587 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.243798 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:14.244459 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.246617 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:14.251666 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:14.258227 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.055s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:14.408798 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Pushing event updated for resources: {'Network': ['ID=19c09cdd-9420-4946-a105-024d5ea86ec3,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:14.411502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.160s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:14.423259 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:54:14.427872 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.441086 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.448934 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Pushing event updated for resources: {'Network': ['ID=36cce22a-502c-464d-a9ac-50f2bc247e07,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:14.452484 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.194s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:14.494501 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.501808 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 889 time: 1.1006296 May 31 11:54:14.504669 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 56998) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:14.511384 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Request body: {'subnet': {'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-subnet', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:14.518486 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:14.523029 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Usage tracker for resource:subnet and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:14.528882 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:14.529529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.531585 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 1.6927676 May 31 11:54:14.532335 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.535415 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57002) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:14.539141 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:14.541276 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:14.568282 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Request body: {'subnet': {'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeTestJSON-1240663657-subnet', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:14.578742 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:14.590445 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:54:14.599364 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Usage tracker for resource:subnet and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:14.602434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.607454 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:14.610463 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] vlan segment allocate from pool success with {'vlan_id': 392, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:54:14.614084 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Made reservation on behalf of 0e7f0a333ee54b31bcd5c2290c3e284d for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:14.617095 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.629399 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] vlan segment allocate from pool success with {'vlan_id': 356, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:54:14.643145 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.643482 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Added segment a846ec5a-b6ef-4a36-83ff-119eb454231c of type vlan for network 709626b6-807b-4256-9031-1578948f1fe8 May 31 11:54:14.674119 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.686636 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:14.693134 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=45609a47-7b41-4914-8efd-807ab69ca2c6,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:14.699211 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] vlan segment allocate from pool success with {'vlan_id': 387, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:54:14.733704 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.734046 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Added segment f3279336-e17b-42d0-9e25-1cefbd5c68b4 of type vlan for network 02ae88e7-d3df-4dfe-bee0-364625c3e776 May 31 11:54:14.762022 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.762382 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Added segment 8a1730f3-5150-4597-9255-89922dd429cc of type vlan for network c36bb4a9-2822-49b1-b291-69604a881999 May 31 11:54:14.783805 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9430886b-049d-489c-b4b2-fe5fa25dfff3,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:14.789504 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Made reservation on behalf of b866e6eab1b04406a7280f91698983ce for: {'subnet': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:14.792055 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.806078 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.270s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:14.815298 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.237s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:14.825280 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.953252 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9227c84c-4b80-455f-bb10-219156b82a20,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:14.960439 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:14.967770 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.153s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:14.968042 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.276s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:14.968273 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:15.074024 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.074742 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:15.075314 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.082356 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:15.180966 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.181868 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.194739 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:15.195870 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.241244 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:15.241435 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.259963 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Pushing event updated for resources: {'Network': ['ID=709626b6-807b-4256-9031-1578948f1fe8,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:15.271172 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.189s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:15.272223 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 709626b6-807b-4256-9031-1578948f1fe8 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:54:15.274109 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-99431086-aa22-41c7-97b5-2cd367b82465 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.064s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:15.382259 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 892 time: 2.0961087 May 31 11:54:15.383702 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 888 time: 2.0197418 May 31 11:54:15.383833 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57042) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:15.387507 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57046) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:15.401509 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Pushing event updated for resources: {'Network': ['ID=c36bb4a9-2822-49b1-b291-69604a881999,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:15.401827 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Request body: {'subnet': {'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999', 'cidr': '172.20.0.0/28', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-subnet', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:15.418065 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network c36bb4a9-2822-49b1-b291-69604a881999 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:54:15.425145 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Request body: {'subnet': {'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeNegativeTest-1594415552-subnet', 'tenant_id': '7dea63b6c4a340d7b361fe91ca6b8281', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:15.432396 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 885 time: 2.1003914 May 31 11:54:15.434826 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:15.435545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57052) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:15.441069 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Usage tracker for resource:subnet and tenant:c94978b38250481095d2824adfb16ed4 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:15.446988 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:15.452423 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:15.461813 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:15.464087 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Usage tracker for resource:subnet and tenant:7dea63b6c4a340d7b361fe91ca6b8281 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:15.466751 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.468540 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:15.480809 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c4bb6b22-efff-4ea5-ae28-75ebc1c56be7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:15.483427 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Request body: {'subnet': {'network_id': '02ae88e7-d3df-4dfe-bee0-364625c3e776', 'cidr': '172.20.0.0/28', 'name': 'tempest-DeleteServersTestJSON-1571582306-subnet', 'tenant_id': '3b87c0540dea485d8ec60a8628fced11', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:15.512146 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:15.530007 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:15.538188 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Usage tracker for resource:subnet and tenant:3b87c0540dea485d8ec60a8628fced11 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:15.545403 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:15.548582 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Pushing event updated for resources: {'Network': ['ID=02ae88e7-d3df-4dfe-bee0-364625c3e776,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:15.555823 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.281s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:15.556545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.297s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:15.556974 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.001s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:15.558267 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:15.587140 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 02ae88e7-d3df-4dfe-bee0-364625c3e776 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:54:15.596432 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Checking segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:15.596872 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Network hostable DHCP agents. Network: 19c09cdd-9420-4946-a105-024d5ea86ec3, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:54:15.612035 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Pushing event updated for resources: {'Subnet': ['ID=4c55d678-c52f-4b1e-b005-86edad0fbf12,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:15.614096 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Made reservation on behalf of 7dea63b6c4a340d7b361fe91ca6b8281 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:15.616659 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Notify callbacks [] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.640291 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.160s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:15.659804 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Made reservation on behalf of c94978b38250481095d2824adfb16ed4 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:15.659804 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.742853 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Made reservation on behalf of 3b87c0540dea485d8ec60a8628fced11 for: {'subnet': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:15.745542 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Notify callbacks [] for subnet, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.859732 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57064) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:15.861183 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:15.869775 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:15.936049 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57070) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:15.967234 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Checking segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:15.967908 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Network hostable DHCP agents. Network: 36cce22a-502c-464d-a9ac-50f2bc247e07, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99040) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:54:15.975588 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Pushing event updated for resources: {'Subnet': ['ID=1b74adba-cf2f-4db5-ad92-27ea39726767,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:15.982789 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:15.988765 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:54:16.067117 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99040) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:54:16.232026 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.240525 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 19c09cdd-9420-4946-a105-024d5ea86ec3) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:16.250854 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 36cce22a-502c-464d-a9ac-50f2bc247e07) {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:16.261278 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:16.261278 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.261546 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:16.261546 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.261728 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1c6ee6f8-e433-43b2-8e13-ad7ee4d5ba91 None None] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:16.284660 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:16.295807 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] Usage tracker for resource:security_group_rule and tenant:53601515be894103856403d39a92f040 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:16.305172 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:16.308137 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.348570 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f1c8a436-7670-4491-83a5-327686c67934 None None] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:16.377825 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 1.8184648 May 31 11:54:16.379703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57082) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:16.381119 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:16.386636 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:16.394085 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:16.398960 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:16.399910 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Usage tracker for resource:router and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:16.404570 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:16.414205 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:16.417387 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Usage tracker for resource:security_group_rule and tenant:8a18d2911fe147ed8206f370f844f204 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:16.421038 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:16.442355 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:16.472867 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.478774 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-154171be-3982-4e32-b325-af908fb303da admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:16.482745 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 852 time: 1.9742503 May 31 11:54:16.483909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57086) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:16.504234 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:16.518483 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:16.534153 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Usage tracker for resource:router and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:16.540146 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:16.556191 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:16.563510 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.564597 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.570782 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Pushing event updated for resources: {'Subnet': ['ID=94658dd6-8c5b-487d-9286-e8ccb8e389f4,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:16.578004 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:16.589749 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Checking segment: {'id': 'f3279336-e17b-42d0-9e25-1cefbd5c68b4', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 392, 'network_id': '02ae88e7-d3df-4dfe-bee0-364625c3e776'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:16.590269 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Network hostable DHCP agents. Network: 02ae88e7-d3df-4dfe-bee0-364625c3e776, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99040) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:54:16.603155 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.679530 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.680498 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.703007 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.723484 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-19c09cdd-9420-4946-a105-024d5ea86ec3', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 11:54:16.724813 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.737802 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-36cce22a-502c-464d-a9ac-50f2bc247e07', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 11:54:16.738618 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.745442 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.747398 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.783935 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.798483 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6cc599f6-8deb-49b2-9d34-2eb3485499d3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:16.843532 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Network 02ae88e7-d3df-4dfe-bee0-364625c3e776 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99040) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:54:16.855835 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.879767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.934133 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:16.938578 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=53601515be894103856403d39a92f040&name=default HTTP/1.1" status: 200 len: 2975 time: 1.0015883 May 31 11:54:16.941034 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Checking segment: {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:16.941376 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Network hostable DHCP agents. Network: c36bb4a9-2822-49b1-b291-69604a881999, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:54:16.943829 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57092) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:16.960166 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Pushing event updated for resources: {'Subnet': ['ID=6312c1ab-b503-4bcc-af02-8274dbf2679c,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:16.987666 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.189s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:16.988803 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.098s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:17.013868 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Allocated IP 172.20.0.2 (19c09cdd-9420-4946-a105-024d5ea86ec3/4c55d678-c52f-4b1e-b005-86edad0fbf12/02e88ea0-e503-4523-95a4-11716e4aff1b) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:17.020934 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Allocated IP 172.20.0.2 (36cce22a-502c-464d-a9ac-50f2bc247e07/1b74adba-cf2f-4db5-ad92-27ea39726767/5be5e898-142f-455d-ac69-dbbd04f043dc) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:17.086008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 02ae88e7-d3df-4dfe-bee0-364625c3e776) {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:17.115808 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=8a18d2911fe147ed8206f370f844f204&name=default HTTP/1.1" status: 200 len: 2975 time: 1.2533355 May 31 11:54:17.126348 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-dc3dc9f6-14fe-4b42-9831-faba095a2d35 None None] Network 02ae88e7-d3df-4dfe-bee0-364625c3e776 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:17.168784 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:17.170455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.175025 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57096) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:17.184421 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Checking segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:17.184782 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Network hostable DHCP agents. Network: 709626b6-807b-4256-9031-1578948f1fe8, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:54:17.187541 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b02097f7-fa85-4503-9ab3-253b6b6d62b4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a14f22c2-5bdf-464b-9f07-bdc56f887fb2,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:17.229117 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.314641 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a0d88076-1a8a-4c77-9f21-c1671ef2d575 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.322607 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-aa33ab48-d79e-46e8-bb88-89b75ce51c8f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.332506 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Network c36bb4a9-2822-49b1-b291-69604a881999 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:54:17.345058 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a0d88076-1a8a-4c77-9f21-c1671ef2d575 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.481328 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.509350 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, c36bb4a9-2822-49b1-b291-69604a881999) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:17.512271 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Network 709626b6-807b-4256-9031-1578948f1fe8 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:54:17.515354 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.520123 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5c43c0cf-8154-4c45-af63-088d8b18c985,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:17.545326 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:17.545936 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.552337 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a0d88076-1a8a-4c77-9f21-c1671ef2d575 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.554332 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 849 time: 2.1177719 May 31 11:54:17.561044 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 709626b6-807b-4256-9031-1578948f1fe8) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:17.563423 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.576016 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Transition to ACTIVE for port object 5be5e898-142f-455d-ac69-dbbd04f043dc will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:17.581867 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:17.582245 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.586819 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-aa33ab48-d79e-46e8-bb88-89b75ce51c8f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.596553 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Transition to ACTIVE for port object 5be5e898-142f-455d-ac69-dbbd04f043dc will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:17.597928 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:17.601658 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-32c7f6fd-ecac-45ba-9226-590f04932286 None None] Network c36bb4a9-2822-49b1-b291-69604a881999 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:17.618280 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c62b1d89-8430-42f6-a3a6-8e8e3b97f3be None None] Network 709626b6-807b-4256-9031-1578948f1fe8 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:17.620632 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0d88076-1a8a-4c77-9f21-c1671ef2d575 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5c43c0cf-8154-4c45-af63-088d8b18c985,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:17.628190 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a0d88076-1a8a-4c77-9f21-c1671ef2d575 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:17.629287 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a0d88076-1a8a-4c77-9f21-c1671ef2d575 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:17.633951 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a0d88076-1a8a-4c77-9f21-c1671ef2d575 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/5c43c0cf-8154-4c45-af63-088d8b18c985 HTTP/1.1" status: 204 len: 168 time: 0.4577010 May 31 11:54:17.636457 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Transition to ACTIVE for port object 02e88ea0-e503-4523-95a4-11716e4aff1b will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:17.660296 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Transition to ACTIVE for port object 02e88ea0-e503-4523-95a4-11716e4aff1b will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:17.737732 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.741s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:17.737732 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.826s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:17.868634 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Pushing event updated for resources: {'Subnet': ['ID=4c58bc8a-d411-461e-a483-39773ff0d6f4,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:17.887912 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:17.892021 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.932s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:17.892302 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:17.996138 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-3e5d7388-f54b-490b-b19f-ca483a38eb7c admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 852 time: 2.5987759 May 31 11:54:17.996138 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 856 time: 2.6061506 May 31 11:54:17.996483 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57130) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:18.007955 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Request body: {'router': {'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:18.013167 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:18.018425 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Usage tracker for resource:router and tenant:c94978b38250481095d2824adfb16ed4 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:18.024653 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:18.029513 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.030101 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.031528 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:18.071248 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-aa33ab48-d79e-46e8-bb88-89b75ce51c8f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.079543 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.085426 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:18.087910 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.101102 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:18.115379 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aa33ab48-d79e-46e8-bb88-89b75ce51c8f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a14f22c2-5bdf-464b-9f07-bdc56f887fb2,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:18.118030 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-aa33ab48-d79e-46e8-bb88-89b75ce51c8f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.119353 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-aa33ab48-d79e-46e8-bb88-89b75ce51c8f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/a14f22c2-5bdf-464b-9f07-bdc56f887fb2 HTTP/1.1" status: 204 len: 168 time: 1.1643596 May 31 11:54:18.123250 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-aa33ab48-d79e-46e8-bb88-89b75ce51c8f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:18.125880 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.128045 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.223878 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.233810 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d4ad3a7b-e5bd-47e7-aa16-25bf7ee4fe7c None None] Network c36bb4a9-2822-49b1-b291-69604a881999 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:18.374531 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.411707 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.443777 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Allocated IP 172.27.1.122 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/16b2c21e-cb05-472a-b273-fe85f07e77cc) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:18.459756 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57162) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:18.577447 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:18.578304 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-28a14d4a-4ab3-4235-a1f0-aa9ea28e8f6a None None] Network 02ae88e7-d3df-4dfe-bee0-364625c3e776 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:18.622436 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.634591 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:18.656594 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:18.658026 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:18.659884 dv-cinder-791239-4-iscsi neutron-server[99039]: WARNING keystonemiddleware.auth_token [-] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 31 11:54:18.662502 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Pushing event updated for resources: {'Port': ['ID=5be5e898-142f-455d-ac69-dbbd04f043dc,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:18.715435 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-468be5b3-aa32-4402-bc4c-73384a8a82db None None] Network 709626b6-807b-4256-9031-1578948f1fe8 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:18.727912 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:18.740602 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.744646 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:18.748191 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:18.748336 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.748814 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.748931 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.749178 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.749488 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.749831 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.750147 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.750456 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.750787 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.751096 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.751401 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.751701 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Tracked resource versions report: May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: pushed versions: May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: { 'AddressGroup': {'1.2'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'ConntrackHelper': {'1.0'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Log': {'1.0'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Network': {'1.1'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Port': {'1.7'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'PortForwarding': {'1.2'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'QosPolicy': {'1.8'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SecurityGroup': {'1.4'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SecurityGroupRule': {'1.2'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SubPort': {'1.0'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Subnet': {'1.1'}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Trunk': {'1.1'}} May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: consumer versions: May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: { DHCP agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: HyperV agent@hv-cinder-79123: {}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: L3 agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: Metadata agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: Metering agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: Open vSwitch agent@dv-cinder-791239-4-iscsi: { 'AddressGroup': '1.2', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'ConntrackHelper': '1.0', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Log': '1.0', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Network': '1.1', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Port': '1.7', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'PortForwarding': '1.2', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'QosPolicy': '1.8', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SecurityGroup': '1.4', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SecurityGroupRule': '1.2', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'SubPort': '1.0', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Subnet': '1.1', May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: 'Trunk': '1.1'}} May 31 11:54:18.752672 dv-cinder-791239-4-iscsi neutron-server[99041]: {{(pid=99041) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 31 11:54:18.856292 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57166) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:18.907055 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:18.911732 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.913301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:18.918800 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:18.935391 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:18.956568 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Allocated IP 172.27.1.185 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/b2b561e9-692d-4b72-911d-dd2b5feeee1c) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:19.002709 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:19.066444 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Usage tracker for resource:security_group_rule and tenant:c2d9e876b3264c0fa7cfcf80ebef3dd3 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:19.069118 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Pushing event updated for resources: {'Port': ['ID=16b2c21e-cb05-472a-b273-fe85f07e77cc,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:19.079673 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:19.082455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.334s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:19.098963 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Pushing event updated for resources: {'Port': ['ID=02e88ea0-e503-4523-95a4-11716e4aff1b,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:19.100148 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:19.115415 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.014s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:19.121196 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.627s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:19.121820 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.001s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:19.126940 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Attempting to bind port 02e88ea0-e503-4523-95a4-11716e4aff1b on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:19.127308 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Attempting to bind port 02e88ea0-e503-4523-95a4-11716e4aff1b by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:19.127757 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Attempting to bind port 02e88ea0-e503-4523-95a4-11716e4aff1b on network 19c09cdd-9420-4946-a105-024d5ea86ec3 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:19.129824 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Attempting to bind port 5be5e898-142f-455d-ac69-dbbd04f043dc on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:19.130178 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Attempting to bind port 5be5e898-142f-455d-ac69-dbbd04f043dc by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:19.130520 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Attempting to bind port 5be5e898-142f-455d-ac69-dbbd04f043dc on network 36cce22a-502c-464d-a9ac-50f2bc247e07 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:19.179085 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 53, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:19.179517 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Checking segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:19.179850 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.180238 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Bound using segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:19.180775 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Bound port: 02e88ea0-e503-4523-95a4-11716e4aff1b, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:19.181150 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.198331 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 53, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:19.199265 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Checking segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:19.199488 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.199837 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Bound using segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:19.200327 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Bound port: 5be5e898-142f-455d-ac69-dbbd04f043dc, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:19.200734 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.205929 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-340d044c-5800-422f-83c2-03ee28b04475 req-8ad73216-c209-42ee-afba-a735a2ae1438 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:19.214996 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-8ad73216-c209-42ee-afba-a735a2ae1438 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/networks?id=02ae88e7-d3df-4dfe-bee0-364625c3e776 HTTP/1.1" status: 200 len: 812 time: 0.7549219 May 31 11:54:19.236089 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-fa9dea0d-0279-4f73-aeba-58adee33fc21 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/quotas/3b87c0540dea485d8ec60a8628fced11 HTTP/1.1" status: 200 len: 375 time: 0.0198193 May 31 11:54:19.321218 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-c9454121-5c38-4c50-bccd-4fb352539bec tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=3b87c0540dea485d8ec60a8628fced11&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0837307 May 31 11:54:19.438825 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.523620 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.532754 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] For port 5be5e898-142f-455d-ac69-dbbd04f043dc, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:19.535998 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] For port 02e88ea0-e503-4523-95a4-11716e4aff1b, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:19.543039 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] For port 5be5e898-142f-455d-ac69-dbbd04f043dc, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=5be5e898-142f-455d-ac69-dbbd04f043dc,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:19.543039 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] For port 02e88ea0-e503-4523-95a4-11716e4aff1b, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=02e88ea0-e503-4523-95a4-11716e4aff1b,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:19.594340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.636759 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a0d88076-1a8a-4c77-9f21-c1671ef2d575 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:19.643693 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Synchronizing usage tracker for tenant:cf8b125186c446b2aec91a3f69c5f1fa on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:19.649110 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.653235 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.672754 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.005s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:19.675891 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.685351 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:19.770781 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Unset dirty status for tenant:cf8b125186c446b2aec91a3f69c5f1fa on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:19.775316 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=c2d9e876b3264c0fa7cfcf80ebef3dd3&name=default HTTP/1.1" status: 200 len: 2975 time: 0.9057956 May 31 11:54:19.776070 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b1b5e8ff-51c6-47fd-abb5-8a62f6173c1d,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:19.801440 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Pushing event updated for resources: {'Port': ['ID=02e88ea0-e503-4523-95a4-11716e4aff1b,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:19.803074 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57184) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:19.806057 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:19.852120 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.215s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:19.855242 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57188) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:19.927268 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-1a8bc304-2fc0-4d0f-b650-c65b88073c26 service neutron] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99039) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:54:19.928399 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-1a8bc304-2fc0-4d0f-b650-c65b88073c26 service neutron] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.2687359 May 31 11:54:19.934156 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Pushing event updated for resources: {'Port': ['ID=5be5e898-142f-455d-ac69-dbbd04f043dc,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:19.944412 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.272s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:19.944826 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.185s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:19.945344 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.001s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:20.035855 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57204) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:20.035855 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.037630 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:20.044324 dv-cinder-791239-4-iscsi neutron-server[99040]: WARNING keystonemiddleware.auth_token [-] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 31 11:54:20.081676 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-304a9669-5f9a-4234-95ac-0056252d6336 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.112090 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:20.114201 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:20.144484 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:20.209008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Usage tracker for resource:security_group_rule and tenant:a032605d6b54453d828d63d7132e8fc7 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:20.219411 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:20.236727 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-304a9669-5f9a-4234-95ac-0056252d6336 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.251602 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Pushing event updated for resources: {'Port': ['ID=b2b561e9-692d-4b72-911d-dd2b5feeee1c,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:20.296271 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:20.382304 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.577s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:20.383340 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.429329 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.503445 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-585c4210-5ac6-4834-9881-f7d4ceda67ed service neutron] 10.222.0.48 "GET /v2.0/networks/02ae88e7-d3df-4dfe-bee0-364625c3e776?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5646338 May 31 11:54:20.546719 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-72bc0cc6-8726-418c-9e2e-41890a2d30f8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:20.546971 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-72bc0cc6-8726-418c-9e2e-41890a2d30f8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:network {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:20.592390 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.665125 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-304a9669-5f9a-4234-95ac-0056252d6336 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.692545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:20.724172 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-304a9669-5f9a-4234-95ac-0056252d6336 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.725285 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-304a9669-5f9a-4234-95ac-0056252d6336 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/b1b5e8ff-51c6-47fd-abb5-8a62f6173c1d HTTP/1.1" status: 204 len: 168 time: 0.9210596 May 31 11:54:20.736130 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-304a9669-5f9a-4234-95ac-0056252d6336 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b1b5e8ff-51c6-47fd-abb5-8a62f6173c1d,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:20.902825 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-304a9669-5f9a-4234-95ac-0056252d6336 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.210s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:20.909325 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-e4300100-4ede-4180-9d9d-050924b5ecd6 service neutron] 10.222.0.48 "GET /v2.0/networks/02ae88e7-d3df-4dfe-bee0-364625c3e776?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4050288 May 31 11:54:20.911453 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6b2d4fe2-2888-47c1-9f3e-eb8ccb8a2905 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:20.912203 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Allocated IP 172.27.1.64 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/3778e8e5-5392-49be-a3b1-1b01bb388e8d) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:20.941326 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:21.036553 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:21.063092 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:21.077765 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:21.115753 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=a032605d6b54453d828d63d7132e8fc7&name=default HTTP/1.1" status: 200 len: 2975 time: 1.2532325 May 31 11:54:21.119062 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57214) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:21.123388 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Router 6c5aae18-4faa-4947-af35-4078d428c2b9 is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99040) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 11:54:21.134802 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c83a18e4-6fb5-4ec5-b18f-a93b4f170439,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:21.140663 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:21.228534 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-72bc0cc6-8726-418c-9e2e-41890a2d30f8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:network {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:21.234919 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-72bc0cc6-8726-418c-9e2e-41890a2d30f8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 815 time: 1.2272408 May 31 11:54:21.294478 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0d66488f-ae4c-4e82-969b-090c0b24fe4b None None] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:21.333126 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-25edd7b4-0fae-4e5a-badf-a891fe78cdf6 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/quotas/7dea63b6c4a340d7b361fe91ca6b8281 HTTP/1.1" status: 200 len: 375 time: 0.0971010 May 31 11:54:21.467970 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-4ad4420c-ec88-4a7e-87fd-9c81a3cdc573 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&fields=id HTTP/1.1" status: 200 len: 210 time: 0.1326072 May 31 11:54:21.574045 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a1fef88c-8b7f-410b-be21-88e1db7a945f req-36109148-0e35-47df-a846-25c958f908a8 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.1946042 May 31 11:54:21.597220 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-b34b13c3-a5f5-492d-9dae-778908394a0d None None] Provisioning for port 5be5e898-142f-455d-ac69-dbbd04f043dc completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:21.609455 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:21.772980 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c9baca86-a559-4b8f-a630-df1dc908f80d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:21.814994 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d7fcc384-bf15-4ad3-b404-1bb510618228 None None] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:54:21.890477 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:54:21.900141 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c9baca86-a559-4b8f-a630-df1dc908f80d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:21.922909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:21.924340 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Sync routers for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:54:21.946789 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:21.948766 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:53:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:22.030480 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 02e88ea0-e503-4523-95a4-11716e4aff1b up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:22.105430 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:22.123069 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Notify callbacks [] for router, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:22.124313 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 923 time: 5.7436936 May 31 11:54:22.127119 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57264) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:22.132717 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:22.261103 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-63e920f1-1fcd-48bc-8800-c6244c6c18a0 None None] Provisioning for port 02e88ea0-e503-4523-95a4-11716e4aff1b completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:22.269720 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-e2021bf8-9192-4ab3-b7c4-46c5dd68f223 service neutron] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99040) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:54:22.270706 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-e2021bf8-9192-4ab3-b7c4-46c5dd68f223 service neutron] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.4299970 May 31 11:54:22.279147 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c9baca86-a559-4b8f-a630-df1dc908f80d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:22.306016 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:22.332909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:22.334106 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9baca86-a559-4b8f-a630-df1dc908f80d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c83a18e4-6fb5-4ec5-b18f-a93b4f170439,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:22.406784 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 02e88ea0-e503-4523-95a4-11716e4aff1b completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:22.433161 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c9baca86-a559-4b8f-a630-df1dc908f80d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:22.433312 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 02e88ea0-e503-4523-95a4-11716e4aff1b triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:22.433981 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:22.471678 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:22.511699 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:22.514504 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:22.543359 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c9baca86-a559-4b8f-a630-df1dc908f80d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:22.544877 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c9baca86-a559-4b8f-a630-df1dc908f80d admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/c83a18e4-6fb5-4ec5-b18f-a93b4f170439 HTTP/1.1" status: 204 len: 168 time: 1.4239051 May 31 11:54:22.692671 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-8b587389-6300-4f18-ba48-0667a2ce0350 service neutron] 10.222.0.48 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4123886 May 31 11:54:22.723192 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Pushing event updated for resources: {'Port': ['ID=3778e8e5-5392-49be-a3b1-1b01bb388e8d,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:22.730498 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.596s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:22.852519 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:22.935072 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 02e88ea0-e503-4523-95a4-11716e4aff1b, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=02e88ea0-e503-4523-95a4-11716e4aff1b,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:22.991506 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:22.992745 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 5be5e898-142f-455d-ac69-dbbd04f043dc up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:23.023959 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:19Z', 'revision_number': 3, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:18Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:23.029342 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Checking router: 6c5aae18-4faa-4947-af35-4078d428c2b9 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:54:23.029853 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Updating router 6c5aae18-4faa-4947-af35-4078d428c2b9 port 16b2c21e-cb05-472a-b273-fe85f07e77cc binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:54:23.066192 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-35b077b1-adb7-4c5f-8712-6ed52c585117 req-a98b8316-70b1-42cc-a11e-de2fadef9151 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.1836226 May 31 11:54:23.121058 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:23.125097 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:23.379534 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:23.386344 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=02e88ea0-e503-4523-95a4-11716e4aff1b,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:23.393344 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.272s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:23.407722 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-de427c71-b996-41e6-a008-4e19dbaf01eb None None] Provisioning complete for port 02e88ea0-e503-4523-95a4-11716e4aff1b triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:23.408482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-de427c71-b996-41e6-a008-4e19dbaf01eb None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:23.437771 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-06c02b40-fbb0-4cce-a4cb-49b7cc266ad6 service neutron] 10.222.0.48 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5989573 May 31 11:54:23.476174 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99039) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 11:54:23.613626 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 5be5e898-142f-455d-ac69-dbbd04f043dc completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:23.646115 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 5be5e898-142f-455d-ac69-dbbd04f043dc triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:23.649966 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:23.779351 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-0655d329-a48c-4bf3-aed8-7aaa3372fc47 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:23.848888 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:23.905319 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Transition to ACTIVE for port object 16b2c21e-cb05-472a-b273-fe85f07e77cc will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:23.979944 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:23.985195 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:24.065520 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.172056 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.226791 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Pushing event updated for resources: {'Port': ['ID=16b2c21e-cb05-472a-b273-fe85f07e77cc,revision_number=2']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:24.229220 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Attempting to bind port 16b2c21e-cb05-472a-b273-fe85f07e77cc on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:24.229836 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Attempting to bind port 16b2c21e-cb05-472a-b273-fe85f07e77cc by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:24.230328 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Attempting to bind port 16b2c21e-cb05-472a-b273-fe85f07e77cc on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:24.247987 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:24.254857 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:24.254857 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.254857 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:24.254857 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Bound port: 16b2c21e-cb05-472a-b273-fe85f07e77cc, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:24.255249 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9192e8cc-14ad-4073-ae24-a52581ae199f req-aaad98c2-c756-4842-8365-3ed3c8d2c2d6 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.0670564 May 31 11:54:24.266785 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.279610 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:54:24.293605 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Sync routers for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:54:24.324363 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 5be5e898-142f-455d-ac69-dbbd04f043dc, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=5be5e898-142f-455d-ac69-dbbd04f043dc,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:24.331241 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:24.331901 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:24.342666 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.343012 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.343314 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.343647 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.343948 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.344268 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.344579 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.344886 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.345215 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.345526 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.345859 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.346164 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Tracked resource versions report: May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: pushed versions: May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: { 'AddressGroup': {'1.2'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'ConntrackHelper': {'1.0'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Log': {'1.0'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Network': {'1.1'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Port': {'1.7'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'PortForwarding': {'1.2'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'QosPolicy': {'1.8'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SecurityGroup': {'1.4'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SecurityGroupRule': {'1.2'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SubPort': {'1.0'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Subnet': {'1.1'}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Trunk': {'1.1'}} May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: consumer versions: May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: { DHCP agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: HyperV agent@hv-cinder-79123: {}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: L3 agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: Metadata agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: Metering agent@dv-cinder-791239-4-iscsi: {}, May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: Open vSwitch agent@dv-cinder-791239-4-iscsi: { 'AddressGroup': '1.2', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'ConntrackHelper': '1.0', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Log': '1.0', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Network': '1.1', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Port': '1.7', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'PortForwarding': '1.2', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'QosPolicy': '1.8', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SecurityGroup': '1.4', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SecurityGroupRule': '1.2', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'SubPort': '1.0', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Subnet': '1.1', May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: 'Trunk': '1.1'}} May 31 11:54:24.347146 dv-cinder-791239-4-iscsi neutron-server[99042]: {{(pid=99042) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 31 11:54:24.356243 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.371s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:24.367100 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.395322 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ca11f16f-d11b-49df-bc35-0403d31d38d2 req-ffdccde5-58d7-48df-97c1-c3b72d89d7c4 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.1328981 May 31 11:54:24.461591 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] For port 16b2c21e-cb05-472a-b273-fe85f07e77cc, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:24.465840 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] For port 16b2c21e-cb05-472a-b273-fe85f07e77cc, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=16b2c21e-cb05-472a-b273-fe85f07e77cc,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:24.470124 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.500227 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-b3bd3cf9-e0a8-4e43-820f-50c385a00907 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:24.571826 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.583590 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:24.733500 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:24.733500 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:24.746569 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.750908 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-0d258858-48c5-4b69-be39-5e1635620b37 admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 928 time: 8.2637136 May 31 11:54:24.751171 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57414) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:24.787963 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 02e88ea0-e503-4523-95a4-11716e4aff1b up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:24.875571 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Allocated IP 172.20.0.1 (36cce22a-502c-464d-a9ac-50f2bc247e07/1b74adba-cf2f-4db5-ad92-27ea39726767/b5af8b49-c5f1-494f-aa5a-3d080497d163) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:24.920241 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.941388 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=5be5e898-142f-455d-ac69-dbbd04f043dc,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:24.951311 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.451s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:24.968869 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 02e88ea0-e503-4523-95a4-11716e4aff1b triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:24.969509 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:24.991085 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:25.009926 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:25.010975 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:25.166006 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Pushing event updated for resources: {'Port': ['ID=16b2c21e-cb05-472a-b273-fe85f07e77cc,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:25.178343 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.594s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:25.460820 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:19Z', 'revision_number': 3, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:24Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:25.468422 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-32063992-e6a8-44aa-9dee-3137f37c6b3f None None] The sync data for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi is: [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:19Z', 'revision_number': 3, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:24Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:54:25.632767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Router 402aeb18-0749-46be-8b62-c6de2c58aad6 is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99039) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 11:54:25.656213 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2c29a036-0f28-4ded-9516-f20cad2ec378 req-b8ff4cf0-1e7b-4fa4-9547-3e58d81fc7e2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.1777358 May 31 11:54:25.742439 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:25.759652 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Transition to ACTIVE for port object b5af8b49-c5f1-494f-aa5a-3d080497d163 will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:25.943132 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:20Z', 'revision_number': 3, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:18Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:25.971178 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Checking router: 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:54:25.976557 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Updating router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 port b2b561e9-692d-4b72-911d-dd2b5feeee1c binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:54:26.001925 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7c478527-34be-469e-85b5-9b21677c8d46 req-8f6a6b7c-ffb3-48d2-9a7c-637083946b80 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.3317637 May 31 11:54:26.057730 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:26.069109 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:26.076453 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:54:26.088409 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Sync routers for ids ['402aeb18-0749-46be-8b62-c6de2c58aad6'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:54:26.107347 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Routers {'402aeb18-0749-46be-8b62-c6de2c58aad6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:26.107829 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['402aeb18-0749-46be-8b62-c6de2c58aad6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:26.130933 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:26.239123 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:26.242766 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:26.253995 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 930 time: 8.2419038 May 31 11:54:26.257213 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:26.257213 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:26.258524 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57464) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:26.268304 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:26.269380 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:26.312638 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Pushing event updated for resources: {'Port': ['ID=b5af8b49-c5f1-494f-aa5a-3d080497d163,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:26.328485 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.256s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:26.624239 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.47', 49549) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:26.871821 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 5be5e898-142f-455d-ac69-dbbd04f043dc up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:26.975539 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Set router 402aeb18-0749-46be-8b62-c6de2c58aad6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:27.155404 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:27.182892 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Transition to ACTIVE for port object b2b561e9-692d-4b72-911d-dd2b5feeee1c will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:27.207230 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-df98522e-3140-4de0-a929-6c83ada29344 None None] Provisioning for port b5af8b49-c5f1-494f-aa5a-3d080497d163 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:27.228278 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-df98522e-3140-4de0-a929-6c83ada29344 None None] Provisioning complete for port b5af8b49-c5f1-494f-aa5a-3d080497d163 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:27.228590 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-df98522e-3140-4de0-a929-6c83ada29344 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:27.304268 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b605d576-0a9a-4ce9-a615-36ca25afe12a req-7738fd4f-ead8-4ca7-bdbd-9119a8fc58e8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.1750171 May 31 11:54:27.338992 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:27.360466 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-340d044c-5800-422f-83c2-03ee28b04475 req-9b5ffd60-4f5d-4fc4-ba74-01cd02d2c541 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:27.366054 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-340d044c-5800-422f-83c2-03ee28b04475 req-9b5ffd60-4f5d-4fc4-ba74-01cd02d2c541 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Synchronizing usage tracker for tenant:3b87c0540dea485d8ec60a8628fced11 on resource:network {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:27.398109 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:27.481022 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-df98522e-3140-4de0-a929-6c83ada29344 None None] Port b5af8b49-c5f1-494f-aa5a-3d080497d163 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:54:27.514595 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Attempting to bind port b2b561e9-692d-4b72-911d-dd2b5feeee1c on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:27.514850 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Attempting to bind port b2b561e9-692d-4b72-911d-dd2b5feeee1c by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:27.515342 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Attempting to bind port b2b561e9-692d-4b72-911d-dd2b5feeee1c on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:27.524924 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:27.525301 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:27.525775 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:27.526244 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:27.526868 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Bound port: b2b561e9-692d-4b72-911d-dd2b5feeee1c, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:27.527336 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:27.558057 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:23Z', 'revision_number': 3, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:20Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:27.574201 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-013cf3e9-7f05-4751-ae54-a8f89d18e71e req-8799e600-85b5-4ca7-9896-dcc1c5d6e466 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.2072074 May 31 11:54:27.580272 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Checking router: 402aeb18-0749-46be-8b62-c6de2c58aad6 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:54:27.580881 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Updating router 402aeb18-0749-46be-8b62-c6de2c58aad6 port 3778e8e5-5392-49be-a3b1-1b01bb388e8d binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:54:27.650277 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87f8168a-c611-4d19-b7f7-39aedada206f None None] Pushing event updated for resources: {'Port': ['ID=b2b561e9-692d-4b72-911d-dd2b5feeee1c,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:27.670073 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-87f8168a-c611-4d19-b7f7-39aedada206f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.272s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:27.708778 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 5be5e898-142f-455d-ac69-dbbd04f043dc triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:27.709182 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:27.713008 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:28.081779 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:28.089216 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] For port b2b561e9-692d-4b72-911d-dd2b5feeee1c, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:28.089216 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] For port b2b561e9-692d-4b72-911d-dd2b5feeee1c, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b2b561e9-692d-4b72-911d-dd2b5feeee1c,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:28.091378 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-340d044c-5800-422f-83c2-03ee28b04475 req-9b5ffd60-4f5d-4fc4-ba74-01cd02d2c541 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Unset dirty status for tenant:3b87c0540dea485d8ec60a8628fced11 on resource:network {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:28.091378 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-9b5ffd60-4f5d-4fc4-ba74-01cd02d2c541 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.47 "GET /v2.0/networks?id=02ae88e7-d3df-4dfe-bee0-364625c3e776 HTTP/1.1" status: 200 len: 812 time: 1.4075687 May 31 11:54:28.121672 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Request body: {'port': {'device_id': 'd1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4', 'network_id': '02ae88e7-d3df-4dfe-bee0-364625c3e776', 'admin_state_up': True, 'tenant_id': '3b87c0540dea485d8ec60a8628fced11'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:28.127037 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:28.136804 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:28.343186 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 16b2c21e-cb05-472a-b273-fe85f07e77cc up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:28.629005 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:28.669482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:28.670183 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:28.866389 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87f8168a-c611-4d19-b7f7-39aedada206f None None] Pushing event updated for resources: {'Port': ['ID=b2b561e9-692d-4b72-911d-dd2b5feeee1c,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:28.883793 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.47', 49557) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:28.904260 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-87f8168a-c611-4d19-b7f7-39aedada206f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.767s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:28.946624 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:28.990314 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Transition to ACTIVE for port object 3778e8e5-5392-49be-a3b1-1b01bb388e8d will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:29.033455 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5f09650e-77d4-4f2a-ae69-4da9127318ab req-8bd352e4-c47f-4634-8b9a-fb55914aadc3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.3047719 May 31 11:54:29.046961 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57554) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:29.089519 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:29.095038 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:29.116589 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b71c813c-d040-4e5e-8df8-dba3abd981da req-89d534e1-9fef-41b3-ba70-5b4b6664974b tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.3202965 May 31 11:54:29.321941 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:29.330519 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-62061ac9-e43f-4917-963f-82f698d8fa30 None None] Pushing event updated for resources: {'Port': ['ID=3778e8e5-5392-49be-a3b1-1b01bb388e8d,revision_number=2']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:29.378198 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 16b2c21e-cb05-472a-b273-fe85f07e77cc completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:29.413508 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 16b2c21e-cb05-472a-b273-fe85f07e77cc triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:29.420750 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Attempting to bind port 3778e8e5-5392-49be-a3b1-1b01bb388e8d on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:29.422231 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Attempting to bind port 3778e8e5-5392-49be-a3b1-1b01bb388e8d by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:29.422231 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Attempting to bind port 3778e8e5-5392-49be-a3b1-1b01bb388e8d on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:29.428251 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:29.428705 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:29.429038 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:29.429601 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:29.430127 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:29.434239 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Bound port: 3778e8e5-5392-49be-a3b1-1b01bb388e8d, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:29.434852 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:29.505604 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Usage tracker for resource:port and tenant:3b87c0540dea485d8ec60a8628fced11 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:29.515089 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-62061ac9-e43f-4917-963f-82f698d8fa30 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.420s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:29.517955 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:29.547042 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:29.603371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:29.603697 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-3ac7890c-0029-417c-963c-5d9c2f1ae0c8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:29.606359 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-3ac7890c-0029-417c-963c-5d9c2f1ae0c8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 815 time: 0.7051275 May 31 11:54:29.622746 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] For port 3778e8e5-5392-49be-a3b1-1b01bb388e8d, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:29.622746 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] For port 3778e8e5-5392-49be-a3b1-1b01bb388e8d, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=3778e8e5-5392-49be-a3b1-1b01bb388e8d,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:29.632876 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:29.664746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:29.697537 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:29.712791 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Usage tracker for resource:security_group_rule and tenant:a63057a95be546698f4b2472177e3f29 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:29.719463 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:29.721651 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:29.734825 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-ceda27a8-d2bd-4ac4-8336-65656c44979a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:29.765229 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:29.810545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Allocated IP 172.20.0.1 (19c09cdd-9420-4946-a105-024d5ea86ec3/4c55d678-c52f-4b1e-b005-86edad0fbf12/502e1922-1254-4eed-bdc8-4c83694840b3) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:29.903621 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Routers {'402aeb18-0749-46be-8b62-c6de2c58aad6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:29.904436 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['402aeb18-0749-46be-8b62-c6de2c58aad6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:29.936001 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Made reservation on behalf of 3b87c0540dea485d8ec60a8628fced11 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:29.939674 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:30.181707 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Allocated IP 172.20.0.1 (c36bb4a9-2822-49b1-b291-69604a881999/6312c1ab-b503-4bcc-af02-8274dbf2679c/b808043f-df25-4425-a3f4-6ee4d3a2150f) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:30.266868 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:30.360152 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] Set router 402aeb18-0749-46be-8b62-c6de2c58aad6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:30.433432 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-169c821a-80ac-42a0-af7d-9cda770bc4f0 req-48477079-3570-4b50-ba84-9c31b264c956 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.0797443 May 31 11:54:30.437884 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 16b2c21e-cb05-472a-b273-fe85f07e77cc, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=16b2c21e-cb05-472a-b273-fe85f07e77cc,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:30.506899 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:20Z', 'revision_number': 3, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:30.512717 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5820923-7e4b-431a-a646-6aae968f7ead None None] The sync data for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi is: [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:20Z', 'revision_number': 3, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:54:30.522365 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:30.534690 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:30.536828 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:30.555884 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-62061ac9-e43f-4917-963f-82f698d8fa30 None None] Pushing event updated for resources: {'Port': ['ID=3778e8e5-5392-49be-a3b1-1b01bb388e8d,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:30.567760 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5e6f2c0a-544f-40d6-b915-5c6379739991 req-1a24533d-73e2-446e-b990-ec2d21b9139a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.2519403 May 31 11:54:30.581039 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-62061ac9-e43f-4917-963f-82f698d8fa30 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.846s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:30.617526 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:30.666967 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:30.686787 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Allocated IP 172.20.0.8 (02ae88e7-d3df-4dfe-bee0-364625c3e776/94658dd6-8c5b-487d-9286-e8ccb8e389f4/982ed1f8-4257-4da4-9de6-458b5ec1994d) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:30.728371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=16b2c21e-cb05-472a-b273-fe85f07e77cc,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:30.745612 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.209s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:30.894723 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=a63057a95be546698f4b2472177e3f29&name=default HTTP/1.1" status: 200 len: 2979 time: 1.8464983 May 31 11:54:30.924069 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57578) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:30.928614 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57580) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:31.020939 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.061313 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d3d610e7-43fe-45a5-bc62-e3f9290604a2,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:31.069230 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.115536 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:23Z', 'revision_number': 3, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:29Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:31.122989 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fef40c7a-9b0b-4344-9792-0802b9d9f26e None None] The sync data for ids ['402aeb18-0749-46be-8b62-c6de2c58aad6'] in dv-cinder-791239-4-iscsi is: [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:23Z', 'revision_number': 3, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:29Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:54:31.143921 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.477s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:31.177930 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d10c289c-2a1b-40a5-aaf4-26959cb9a400 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.189666 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.234378 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-cf9a83f1-5404-4caf-a137-b707b8c91660 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:31.291805 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Transition to ACTIVE for port object 502e1922-1254-4eed-bdc8-4c83694840b3 will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:31.406701 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d10c289c-2a1b-40a5-aaf4-26959cb9a400 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.441105 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.492140 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.515636 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.524160 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.543292 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d2d17083-ed6d-40d2-9f9d-1a17e83cea3d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:31.594240 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Network c36bb4a9-2822-49b1-b291-69604a881999 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:31.596106 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:31.657168 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:31.660845 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.802068 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:31.925590 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:32.118491 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-36052ff9-ed6d-407f-a7c0-6b05eeae6992 req-4784f2c9-7734-4266-b921-65761b0a75be tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.3968642 May 31 11:54:32.172962 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Pushing event updated for resources: {'Port': ['ID=b5af8b49-c5f1-494f-aa5a-3d080497d163,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:32.182781 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.639s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:32.189454 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-19440599-92f2-4444-a0a8-ddb79bb693a2 req-af729a99-bec7-4d99-be46-d664cd39e64d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.2750916 May 31 11:54:32.195382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:32.196219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b1c16db0-2cd0-4b48-a643-c4f8aeb3b6f4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.081s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:32.205587 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Pushing event updated for resources: {'Port': ['ID=b808043f-df25-4425-a3f4-6ee4d3a2150f,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:32.227703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.993s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:32.266657 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:32.278550 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:32.289345 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Usage tracker for resource:security_group_rule and tenant:a85de8ad6f4f4773acfd828183e9ff4c is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:32.308433 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:32.326613 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:32.327090 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:32.393135 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:32.405428 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:32.445091 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:32.446469 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Pushing event updated for resources: {'Port': ['ID=502e1922-1254-4eed-bdc8-4c83694840b3,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:32.658362 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:32.806302 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.609s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:32.807903 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.063s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:32.831324 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-0d847e92-6845-4464-a15a-b7c5617ed30b None None] Provisioning for port 502e1922-1254-4eed-bdc8-4c83694840b3 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:32.841616 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-0d847e92-6845-4464-a15a-b7c5617ed30b None None] Provisioning complete for port 502e1922-1254-4eed-bdc8-4c83694840b3 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:32.844290 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-0d847e92-6845-4464-a15a-b7c5617ed30b None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:32.903727 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Network 02ae88e7-d3df-4dfe-bee0-364625c3e776 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:32.936312 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 16b2c21e-cb05-472a-b273-fe85f07e77cc up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:32.951055 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d10c289c-2a1b-40a5-aaf4-26959cb9a400 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.027959 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:33.028595 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:33.038477 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:33.048936 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:33.049858 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.056241 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d10c289c-2a1b-40a5-aaf4-26959cb9a400 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d3d610e7-43fe-45a5-bc62-e3f9290604a2,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:33.080848 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d10c289c-2a1b-40a5-aaf4-26959cb9a400 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:33.114413 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.116180 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d10c289c-2a1b-40a5-aaf4-26959cb9a400 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.118291 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d10c289c-2a1b-40a5-aaf4-26959cb9a400 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/d3d610e7-43fe-45a5-bc62-e3f9290604a2 HTTP/1.1" status: 204 len: 168 time: 2.1700268 May 31 11:54:33.118994 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.198165 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-0d847e92-6845-4464-a15a-b7c5617ed30b None None] Port 502e1922-1254-4eed-bdc8-4c83694840b3 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:54:33.224454 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.268553 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 16b2c21e-cb05-472a-b273-fe85f07e77cc triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:33.269204 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.356523 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.445305 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:33.479878 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=a85de8ad6f4f4773acfd828183e9ff4c&name=default HTTP/1.1" status: 200 len: 2979 time: 2.5295060 May 31 11:54:33.497214 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57606) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:33.518061 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-c794a504-3b39-4b18-a06c-dafda2036e97 None None] Provisioning complete for port b5af8b49-c5f1-494f-aa5a-3d080497d163 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:33.518454 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c794a504-3b39-4b18-a06c-dafda2036e97 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.588576 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-c794a504-3b39-4b18-a06c-dafda2036e97 None None] Port b5af8b49-c5f1-494f-aa5a-3d080497d163 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:54:33.719258 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.724475 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b2b561e9-692d-4b72-911d-dd2b5feeee1c up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:33.737503 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e0153c8f-5e0c-468a-908b-a9164a039604 req-b11442d8-a915-4d3c-9649-3b3a77a05526 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.2408941 May 31 11:54:33.800478 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Router 6c5aae18-4faa-4947-af35-4078d428c2b9 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:54:33.819465 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a8e93568-8eeb-4155-9f9a-0d8ae8b8143e,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:33.831606 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.839249 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.384s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:33.864543 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:33.922528 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Pushing event updated for resources: {'Port': ['ID=982ed1f8-4257-4da4-9de6-458b5ec1994d,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:33.942633 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.131s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:34.015116 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.127s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:34.032555 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e0692338-ebe6-46b3-9022-fe0617b3ae7f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.093506 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e0692338-ebe6-46b3-9022-fe0617b3ae7f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.101390 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port b2b561e9-692d-4b72-911d-dd2b5feeee1c completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:34.107535 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port b2b561e9-692d-4b72-911d-dd2b5feeee1c triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:34.108598 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.124843 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 6.0185251 May 31 11:54:34.145730 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Pushing event updated for resources: {'Port': ['ID=502e1922-1254-4eed-bdc8-4c83694840b3,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:34.155618 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.141s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:34.157315 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.252375 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.253395 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.261108 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f441b5cf-2254-4d6c-a861-aff6d4bbd6c1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:34.273781 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70dddc51-778c-4f33-ba65-a5958f4cab1a req-11c9ee7d-e643-4ee7-8b5a-8c6387b14fe3 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:34.280879 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-70dddc51-778c-4f33-ba65-a5958f4cab1a req-11c9ee7d-e643-4ee7-8b5a-8c6387b14fe3 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Synchronizing usage tracker for tenant:3b87c0540dea485d8ec60a8628fced11 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:34.381591 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57612) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:34.401719 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e0692338-ebe6-46b3-9022-fe0617b3ae7f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.421640 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.462335 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.476191 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port b2b561e9-692d-4b72-911d-dd2b5feeee1c, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b2b561e9-692d-4b72-911d-dd2b5feeee1c,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:34.530404 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:34.542738 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:34.544331 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Usage tracker for resource:security_group_rule and tenant:c0a37b3804734995b9334eb0f3b4a2d2 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:34.548600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:34.555688 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e0692338-ebe6-46b3-9022-fe0617b3ae7f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.559576 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-e0692338-ebe6-46b3-9022-fe0617b3ae7f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/a8e93568-8eeb-4155-9f9a-0d8ae8b8143e HTTP/1.1" status: 204 len: 168 time: 1.0497510 May 31 11:54:34.559714 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:54:34.567861 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Sync routers for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:54:34.573434 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] 10.222.0.48 "PUT /v2.0/routers/6c5aae18-4faa-4947-af35-4078d428c2b9/add_router_interface HTTP/1.1" status: 200 len: 503 time: 12.4452581 May 31 11:54:34.574824 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:34.581392 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Pushing event updated for resources: {'Port': ['ID=b808043f-df25-4425-a3f4-6ee4d3a2150f,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:34.608689 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:34.612968 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.355s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:34.613436 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.144s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:34.613851 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:34.651137 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e0692338-ebe6-46b3-9022-fe0617b3ae7f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a8e93568-8eeb-4155-9f9a-0d8ae8b8143e,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:34.695723 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e0692338-ebe6-46b3-9022-fe0617b3ae7f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:34.759028 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-70dddc51-778c-4f33-ba65-a5958f4cab1a req-11c9ee7d-e643-4ee7-8b5a-8c6387b14fe3 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Unset dirty status for tenant:3b87c0540dea485d8ec60a8628fced11 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:34.766500 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-70dddc51-778c-4f33-ba65-a5958f4cab1a req-11c9ee7d-e643-4ee7-8b5a-8c6387b14fe3 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 931 time: 1.1286306 May 31 11:54:34.796547 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Network c36bb4a9-2822-49b1-b291-69604a881999 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:34.800922 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:34.809938 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:34.810410 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:34.859777 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 3778e8e5-5392-49be-a3b1-1b01bb388e8d up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:35.115405 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:35.186287 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Router 402aeb18-0749-46be-8b62-c6de2c58aad6 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:54:35.208038 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-36f1d65f-cc56-417f-a0ff-753adf0c392a req-e4289e78-f5ed-4690-ba4f-2536898571c9 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.0680954 May 31 11:54:35.221150 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=b2b561e9-692d-4b72-911d-dd2b5feeee1c,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:35.236536 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.121s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:35.276856 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-3e083156-70f4-49eb-9fde-4b4ee9708038 None None] Provisioning complete for port 502e1922-1254-4eed-bdc8-4c83694840b3 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:35.277229 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3e083156-70f4-49eb-9fde-4b4ee9708038 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:35.352112 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:35.423112 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:54:35.445293 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-70dddc51-778c-4f33-ba65-a5958f4cab1a req-6376685e-f405-4246-996b-b3d1f8c29bdc tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Synchronizing usage tracker for tenant:3b87c0540dea485d8ec60a8628fced11 on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:35.504206 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 3778e8e5-5392-49be-a3b1-1b01bb388e8d completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:35.520200 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 3778e8e5-5392-49be-a3b1-1b01bb388e8d triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:35.520593 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:35.548521 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:54:35.560628 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Sync routers for ids ['402aeb18-0749-46be-8b62-c6de2c58aad6'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:54:35.564355 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] 10.222.0.48 "PUT /v2.0/routers/402aeb18-0749-46be-8b62-c6de2c58aad6/add_router_interface HTTP/1.1" status: 200 len: 503 time: 9.3035197 May 31 11:54:35.581843 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Routers {'402aeb18-0749-46be-8b62-c6de2c58aad6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:35.582634 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['402aeb18-0749-46be-8b62-c6de2c58aad6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:35.586344 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-3e083156-70f4-49eb-9fde-4b4ee9708038 None None] Port 502e1922-1254-4eed-bdc8-4c83694840b3 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:54:35.844170 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-70dddc51-778c-4f33-ba65-a5958f4cab1a req-6376685e-f405-4246-996b-b3d1f8c29bdc tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Unset dirty status for tenant:3b87c0540dea485d8ec60a8628fced11 on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:35.844565 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-70dddc51-778c-4f33-ba65-a5958f4cab1a req-6376685e-f405-4246-996b-b3d1f8c29bdc tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.7046368 May 31 11:54:36.028315 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.031419 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.039273 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:54:36.073917 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] 10.222.0.48 "PUT /v2.0/routers/8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6/add_router_interface HTTP/1.1" status: 200 len: 503 time: 11.3216763 May 31 11:54:36.110821 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.111750 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 3778e8e5-5392-49be-a3b1-1b01bb388e8d, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=3778e8e5-5392-49be-a3b1-1b01bb388e8d,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:36.113193 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Set router 402aeb18-0749-46be-8b62-c6de2c58aad6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:36.126920 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3b60ad3d-1f22-4978-87d0-e6c9bd7a219e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:36.179817 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Sync routers for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:54:36.240910 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.257086 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:36.295666 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57652) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:36.297618 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=c0a37b3804734995b9334eb0f3b4a2d2&name=default HTTP/1.1" status: 200 len: 2979 time: 1.9082749 May 31 11:54:36.300451 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=124604a9-cc29-483a-b52f-11b01a55ef35,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:36.321430 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.198s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:36.415008 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:36.420139 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:36.483909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1c072cef-6505-4ec1-b26c-527cdef678a3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.767750 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1c072cef-6505-4ec1-b26c-527cdef678a3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.768919 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b2b561e9-692d-4b72-911d-dd2b5feeee1c up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:36.769209 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:29Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b808043f-df25-4425-a3f4-6ee4d3a2150f', 'name': '', 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'mac_address': 'fa:16:3e:4c:b4:7e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_interface', 'standard_attr_id': 101, 'fixed_ips': [{'subnet_id': '6312c1ab-b503-4bcc-af02-8274dbf2679c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:29Z', 'updated_at': '2021-05-31T11:54:33Z', 'revision_number': 2, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'subnets': [{'id': '6312c1ab-b503-4bcc-af02-8274dbf2679c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:36.769209 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Checking router: 402aeb18-0749-46be-8b62-c6de2c58aad6 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:54:36.769209 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Updating router 402aeb18-0749-46be-8b62-c6de2c58aad6 port b808043f-df25-4425-a3f4-6ee4d3a2150f binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:54:36.781096 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=3778e8e5-5392-49be-a3b1-1b01bb388e8d,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:36.784790 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.528s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:36.805311 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.821760 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1c072cef-6505-4ec1-b26c-527cdef678a3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.830525 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:36.857250 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1c072cef-6505-4ec1-b26c-527cdef678a3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=124604a9-cc29-483a-b52f-11b01a55ef35,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:36.863189 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1c072cef-6505-4ec1-b26c-527cdef678a3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:36.882239 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1c072cef-6505-4ec1-b26c-527cdef678a3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:36.883798 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1c072cef-6505-4ec1-b26c-527cdef678a3 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/124604a9-cc29-483a-b52f-11b01a55ef35 HTTP/1.1" status: 204 len: 168 time: 0.5872164 May 31 11:54:36.938915 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57668) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:36.954371 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'security_group': {'name': 'tempest-securitygroup--853597934', 'description': 'tempest-description--1168425090'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:36.970855 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:36.982055 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:security_group and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:36.990512 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:37.011864 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:37.044079 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port b2b561e9-692d-4b72-911d-dd2b5feeee1c triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:37.044079 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.092370 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.127496 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:37.137905 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:37.188360 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:security_group_rule and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:37.201593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:37.212262 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:29Z', 'revision_number': 2, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:37.248010 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6bdfab00-dc95-4305-b7d0-e7ea3367697c req-fee81f59-bd24-40e5-99e3-9d8eee75d359 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.1227190 May 31 11:54:37.250543 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Checking router: 6c5aae18-4faa-4947-af35-4078d428c2b9 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:54:37.250959 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Updating router 6c5aae18-4faa-4947-af35-4078d428c2b9 port b5af8b49-c5f1-494f-aa5a-3d080497d163 binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:54:37.265183 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:37.411864 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.498142 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.498684 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:37.587195 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'SecurityGroup': ['ID=4555883c-aba9-4fc6-894a-6ddaa27e8399,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:37.605310 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:37.618146 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.637146 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.637146 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 0.6954556 May 31 11:54:37.637326 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57686) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:37.641889 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'security_group_rule': {'security_group_id': '4555883c-aba9-4fc6-894a-6ddaa27e8399', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:37.652539 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:37.665703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:security_group_rule and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:37.667867 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:37.675059 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:37.716778 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.735486 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Transition to ACTIVE for port object b808043f-df25-4425-a3f4-6ee4d3a2150f will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:37.771581 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.848865 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:37.855132 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-e811603b-38a8-4228-bf97-25e395eb77b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:38.025765 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.127809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.129497 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:38.175637 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-297bb2d7-0ad5-499d-86e0-0b702be878b3 None None] Pushing event updated for resources: {'Port': ['ID=b808043f-df25-4425-a3f4-6ee4d3a2150f,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:38.233290 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=88e768b9-6955-45f6-8950-9a92c1d7a250,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:38.247614 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:38.269606 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:33Z', 'revision_number': 2, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:38.273609 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-297bb2d7-0ad5-499d-86e0-0b702be878b3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.418s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:38.311160 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.315010 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.6756859 May 31 11:54:38.315338 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57698) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:38.321687 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'security_group_rule': {'security_group_id': '4555883c-aba9-4fc6-894a-6ddaa27e8399', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:38.334722 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Checking router: 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:54:38.338551 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Updating router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 port 502e1922-1254-4eed-bdc8-4c83694840b3 binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:54:38.343809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:38.365390 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:security_group_rule and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:38.370977 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Attempting to bind port b808043f-df25-4425-a3f4-6ee4d3a2150f on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:38.370977 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Attempting to bind port b808043f-df25-4425-a3f4-6ee4d3a2150f by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:38.370977 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Attempting to bind port b808043f-df25-4425-a3f4-6ee4d3a2150f on network c36bb4a9-2822-49b1-b291-69604a881999 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:38.375432 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:38.375586 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:38.375586 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Checking segment: {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:38.375586 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.376611 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Bound using segment: {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:38.376611 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Bound port: b808043f-df25-4425-a3f4-6ee4d3a2150f, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:38.376889 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.393145 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:38.445655 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 3778e8e5-5392-49be-a3b1-1b01bb388e8d up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:38.485425 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] For port b808043f-df25-4425-a3f4-6ee4d3a2150f, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:38.489149 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] For port b808043f-df25-4425-a3f4-6ee4d3a2150f, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b808043f-df25-4425-a3f4-6ee4d3a2150f,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:38.519353 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.537086 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57706) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:38.542986 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'security_group': {'name': 'tempest-securitygroup--569520699', 'description': 'tempest-description--1673445560'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:38.570716 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:38.584255 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:security_group and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:38.595231 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:38.596382 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.610987 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-62061ac9-e43f-4917-963f-82f698d8fa30 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:38.631774 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:38.687844 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 3778e8e5-5392-49be-a3b1-1b01bb388e8d triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:38.688478 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.724645 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Routers {'402aeb18-0749-46be-8b62-c6de2c58aad6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:38.730673 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['402aeb18-0749-46be-8b62-c6de2c58aad6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:38.759076 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.772830 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.804777 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.841091 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:38.851928 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:security_group_rule and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:38.855350 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:38.862438 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:38.930052 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:38.943033 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9d4c5423-c164-4df5-9706-3be2e79b920f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:38.981720 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=9ddfad1e-849e-4835-9f7c-bbd6c867fdb2,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:39.014564 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:39.099265 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] Set router 402aeb18-0749-46be-8b62-c6de2c58aad6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:39.128999 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.133994 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.8142898 May 31 11:54:39.140292 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57728) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:39.149264 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:39.157437 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:39.162226 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:floatingip and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:39.165868 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:39.173274 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:39.235462 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.261294 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f21012f6-9cfa-442f-b326-42af02da004c req-d5425401-3df8-4873-95c3-535447067ca1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:39.262527 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-f21012f6-9cfa-442f-b326-42af02da004c req-d5425401-3df8-4873-95c3-535447067ca1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Synchronizing usage tracker for tenant:c94978b38250481095d2824adfb16ed4 on resource:network {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:39.267784 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.297150 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bf22104-2a4d-4de7-8659-68fa7554554f req-b9f95192-f5eb-4bc7-910f-e570ace6b919 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:39.304374 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3bf22104-2a4d-4de7-8659-68fa7554554f req-b9f95192-f5eb-4bc7-910f-e570ace6b919 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 931 time: 0.6322908 May 31 11:54:39.311594 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.321698 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.331896 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-869ec9c8-6c28-4203-a6e1-764c5dd7cf1a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:39.340849 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Transition to ACTIVE for port object 502e1922-1254-4eed-bdc8-4c83694840b3 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:39.417293 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-297bb2d7-0ad5-499d-86e0-0b702be878b3 None None] Pushing event updated for resources: {'Port': ['ID=b808043f-df25-4425-a3f4-6ee4d3a2150f,revision_number=4']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:39.435507 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-297bb2d7-0ad5-499d-86e0-0b702be878b3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.825s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:39.442251 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3bf22104-2a4d-4de7-8659-68fa7554554f req-7fb7fc8a-9d4e-413e-908f-2c2f1b6417cd tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1334713 May 31 11:54:39.464871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-069a185f-640e-4e1f-8f43-17c4b68f94cd tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99040) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 11:54:39.466717 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-069a185f-640e-4e1f-8f43-17c4b68f94cd tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.47 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0076663 May 31 11:54:39.484695 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:54:39.495697 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'SecurityGroup': ['ID=8e2d5996-59e7-4c4a-9bfd-8eee9c767259,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:39.528744 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.538562 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.207s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:39.553941 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Request body: {'port': {'device_id': 'bb55be8b-198d-40e7-849a-8543fbf35000', 'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'admin_state_up': True, 'tenant_id': '7dea63b6c4a340d7b361fe91ca6b8281'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:39.585740 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-f21012f6-9cfa-442f-b326-42af02da004c req-d5425401-3df8-4873-95c3-535447067ca1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Unset dirty status for tenant:c94978b38250481095d2824adfb16ed4 on resource:network {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:39.589430 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-d5425401-3df8-4873-95c3-535447067ca1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 819 time: 0.5657270 May 31 11:54:39.611556 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:39.662398 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-48115770-447b-4728-a522-2ceba7d99dc8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/quotas/c94978b38250481095d2824adfb16ed4 HTTP/1.1" status: 200 len: 375 time: 0.0379202 May 31 11:54:39.721497 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.751564 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.787856 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:36Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b808043f-df25-4425-a3f4-6ee4d3a2150f', 'name': '', 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'mac_address': 'fa:16:3e:4c:b4:7e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_interface', 'standard_attr_id': 101, 'fixed_ips': [{'subnet_id': '6312c1ab-b503-4bcc-af02-8274dbf2679c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:29Z', 'updated_at': '2021-05-31T11:54:38Z', 'revision_number': 4, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'subnets': [{'id': '6312c1ab-b503-4bcc-af02-8274dbf2679c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:39.797494 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-538d17e4-f5ca-4ccc-b415-56752886c37c None None] The sync data for ids ['402aeb18-0749-46be-8b62-c6de2c58aad6'] in dv-cinder-791239-4-iscsi is: [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:36Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b808043f-df25-4425-a3f4-6ee4d3a2150f', 'name': '', 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'mac_address': 'fa:16:3e:4c:b4:7e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_interface', 'standard_attr_id': 101, 'fixed_ips': [{'subnet_id': '6312c1ab-b503-4bcc-af02-8274dbf2679c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:29Z', 'updated_at': '2021-05-31T11:54:38Z', 'revision_number': 4, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'subnets': [{'id': '6312c1ab-b503-4bcc-af02-8274dbf2679c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:54:39.934035 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:39.940977 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 1.4027505 May 31 11:54:39.949635 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57758) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:39.978394 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ea836161-9a26-4a3f-80cc-13a0eee73c02 req-9b0e33ad-d5d1-4d2e-aac0-e1ff651f1c69 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.1829259 May 31 11:54:39.997589 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'security_group_rule': {'security_group_id': '8e2d5996-59e7-4c4a-9bfd-8eee9c767259', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:40.012409 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:40.096612 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-34bafe81-79cf-4dfd-81c7-7a4c331e1f28 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&fields=id HTTP/1.1" status: 200 len: 255 time: 0.4315357 May 31 11:54:40.101069 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:security_group_rule and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:40.105703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:40.109818 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:40.115388 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:40.120599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Usage tracker for resource:port and tenant:7dea63b6c4a340d7b361fe91ca6b8281 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:40.121717 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:40.138722 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:40.205473 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.210670 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3f4cc04-0212-4978-97d8-90420cca8650 None None] Pushing event updated for resources: {'Port': ['ID=502e1922-1254-4eed-bdc8-4c83694840b3,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:40.237661 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Made reservation on behalf of 7dea63b6c4a340d7b361fe91ca6b8281 for: {'port': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:40.241603 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.267825 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.279869 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-d3f4cc04-0212-4978-97d8-90420cca8650 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.668s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:40.365376 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Transition to ACTIVE for port object b5af8b49-c5f1-494f-aa5a-3d080497d163 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:40.406424 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Attempting to bind port 502e1922-1254-4eed-bdc8-4c83694840b3 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:40.406941 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Attempting to bind port 502e1922-1254-4eed-bdc8-4c83694840b3 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:40.407262 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Attempting to bind port 502e1922-1254-4eed-bdc8-4c83694840b3 on network 19c09cdd-9420-4946-a105-024d5ea86ec3 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:40.432787 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.434955 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:40.435639 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Checking segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:40.436158 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.436616 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Bound using segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:40.437222 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Bound port: 502e1922-1254-4eed-bdc8-4c83694840b3, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:40.438000 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.486531 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Request body: {'port': {'device_id': 'd1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:40.515412 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.538090 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.007s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:40.668377 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-8ff4ad5f-8bc5-4121-9718-2db33ed458ac service neutron] 10.222.0.48 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5708632 May 31 11:54:40.693445 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.704145 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:40.738953 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b808043f-df25-4425-a3f4-6ee4d3a2150f up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:40.739755 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=6b3fa989-8e99-4976-bc63-d883e536e6ce,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:40.761639 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.232s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:40.803265 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.807236 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.8508124 May 31 11:54:40.807606 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Allocated IP 172.20.0.6 (709626b6-807b-4256-9031-1578948f1fe8/4c58bc8a-d411-461e-a483-39773ff0d6f4/c4a233f2-9446-42a7-964d-1dc2496c938a) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:40.820430 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57772) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:40.856076 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'security_group_rule': {'security_group_id': '8e2d5996-59e7-4c4a-9bfd-8eee9c767259', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:40.872792 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] For port 502e1922-1254-4eed-bdc8-4c83694840b3, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:40.881446 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] For port 502e1922-1254-4eed-bdc8-4c83694840b3, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=502e1922-1254-4eed-bdc8-4c83694840b3,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:40.892275 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2670100-fd63-4325-a961-fd3956909425 None None] Pushing event updated for resources: {'Port': ['ID=b5af8b49-c5f1-494f-aa5a-3d080497d163,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:40.892548 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2cc3b356-462b-4470-85ef-e616f4558fbb req-39ed52fc-6f7a-4cc0-b9bf-a2753e6046ea tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:40.910256 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2cc3b356-462b-4470-85ef-e616f4558fbb req-39ed52fc-6f7a-4cc0-b9bf-a2753e6046ea tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 931 time: 0.2440550 May 31 11:54:40.944850 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:40.962175 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:40.991800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:security_group_rule and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:41.014941 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-c2670100-fd63-4325-a961-fd3956909425 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.311s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:41.019799 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:41.033128 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.034435 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Allocated IP 172.27.1.102 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/4f8bed93-6c8d-4aaf-ac82-20f075c5d327) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:41.072443 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:41.080851 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:41.114284 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2cc3b356-462b-4470-85ef-e616f4558fbb req-7b9081a7-1094-44ac-8d1b-e27a72fb9e1b tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2029521 May 31 11:54:41.180801 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57786) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:41.371234 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.523412 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:41.553882 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:41.642766 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-93ddb42d-db83-4944-8b98-2a9a6acfba78 req-e44f2fb8-6f90-45fa-a916-b4db55a57600 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:41.667346 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.692573 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port b808043f-df25-4425-a3f4-6ee4d3a2150f completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:41.702888 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:41.722859 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Usage tracker for resource:security_group_rule and tenant:ebb03233bb6e4db3b613eccaff46cc81 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:41.734444 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:41.755074 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.759242 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:41.774139 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Attempting to bind port b5af8b49-c5f1-494f-aa5a-3d080497d163 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:41.774139 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Attempting to bind port b5af8b49-c5f1-494f-aa5a-3d080497d163 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:41.774533 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Attempting to bind port b5af8b49-c5f1-494f-aa5a-3d080497d163 on network 36cce22a-502c-464d-a9ac-50f2bc247e07 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:41.781766 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3f4cc04-0212-4978-97d8-90420cca8650 None None] Pushing event updated for resources: {'Port': ['ID=502e1922-1254-4eed-bdc8-4c83694840b3,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:41.797721 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:41.798106 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Checking segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:41.798678 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.799080 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Bound using segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:41.799568 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Bound port: b5af8b49-c5f1-494f-aa5a-3d080497d163, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:41.799942 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.804534 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-7f5e4bd8-dd82-4eee-96bc-6258c62b9219 service neutron] 10.222.0.48 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 1.1268733 May 31 11:54:41.809732 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port b808043f-df25-4425-a3f4-6ee4d3a2150f triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:41.810110 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.813096 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-93ddb42d-db83-4944-8b98-2a9a6acfba78 req-e44f2fb8-6f90-45fa-a916-b4db55a57600 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:41.823111 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-93ddb42d-db83-4944-8b98-2a9a6acfba78 req-e44f2fb8-6f90-45fa-a916-b4db55a57600 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.4089386 May 31 11:54:41.839603 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.845826 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:41.847570 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-d3f4cc04-0212-4978-97d8-90420cca8650 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.775s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:41.868430 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:41.902229 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=232a88c7-f5ca-4360-ac9c-201ab1407aa5,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:41.910713 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:41.963334 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.018572 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.077396 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.078584 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.082997 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 1.2614255 May 31 11:54:42.093746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.093746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 57798) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:42.093978 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:42.099868 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:42.105344 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:42.157996 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:42.178523 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:floatingip and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:42.221671 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] For port b5af8b49-c5f1-494f-aa5a-3d080497d163, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:42.226755 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:42.248620 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] For port b5af8b49-c5f1-494f-aa5a-3d080497d163, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b5af8b49-c5f1-494f-aa5a-3d080497d163,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:42.265907 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 982ed1f8-4257-4da4-9de6-458b5ec1994d {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:54:42.279956 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.298423 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:42.315531 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=ebb03233bb6e4db3b613eccaff46cc81&name=default HTTP/1.1" status: 200 len: 2979 time: 1.1284719 May 31 11:54:42.324425 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57814) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:42.329287 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-0b5838ce-3f39-450f-9dc3-93879a4e73ad req-97452a64-9289-4be3-bad6-c5bc9f9a69c9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Synchronizing usage tracker for tenant:c94978b38250481095d2824adfb16ed4 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:42.334911 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.336783 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=70f30481-92c8-4fcb-bd3a-f973cd135e2a,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:42.344785 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.239s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:42.349631 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Network 709626b6-807b-4256-9031-1578948f1fe8 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:42.349631 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.186s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:42.378060 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.407822 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-c2670100-fd63-4325-a961-fd3956909425 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:42.438720 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:42.446884 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:42.452346 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:42.452961 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.469735 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.476891 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Transition to ACTIVE for port object 982ed1f8-4257-4da4-9de6-458b5ec1994d will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:42.479310 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.487745 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:42.538895 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Pushing event updated for resources: {'Port': ['ID=c4a233f2-9446-42a7-964d-1dc2496c938a,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:42.541737 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.193s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:42.630614 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:42.633794 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ae4fb2c2-77f4-42a8-8b8b-c8425c843ec0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.637379 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port b808043f-df25-4425-a3f4-6ee4d3a2150f, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b808043f-df25-4425-a3f4-6ee4d3a2150f,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:42.645321 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.646410 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:42.660495 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:42.665799 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:54:42.666958 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:24Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:42.670333 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-0b5838ce-3f39-450f-9dc3-93879a4e73ad req-97452a64-9289-4be3-bad6-c5bc9f9a69c9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Unset dirty status for tenant:c94978b38250481095d2824adfb16ed4 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:42.671499 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0b5838ce-3f39-450f-9dc3-93879a4e73ad req-97452a64-9289-4be3-bad6-c5bc9f9a69c9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 210 time: 0.5080092 May 31 11:54:42.676675 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 3.2148428 May 31 11:54:42.695017 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Request body: {'port': {'device_id': 'bb55be8b-198d-40e7-849a-8543fbf35000', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:42.811743 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:42.844194 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'Port': ['ID=4f8bed93-6c8d-4aaf-ac82-20f075c5d327,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:42.866557 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.379s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:42.894901 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.060s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:42.948330 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:43.071665 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ae4fb2c2-77f4-42a8-8b8b-c8425c843ec0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:43.078231 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:54:43.163099 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46519af4-1915-44cd-a5e1-32b715e04832 req-0d6053af-edec-4236-9a99-406bea1f9da8 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:43.164952 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-46519af4-1915-44cd-a5e1-32b715e04832 req-0d6053af-edec-4236-9a99-406bea1f9da8 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 931 time: 0.7733662 May 31 11:54:43.171262 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2670100-fd63-4325-a961-fd3956909425 None None] Pushing event updated for resources: {'Port': ['ID=b5af8b49-c5f1-494f-aa5a-3d080497d163,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:43.174581 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-c2670100-fd63-4325-a961-fd3956909425 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.767s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:43.186070 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:43.190633 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-d3f4cc04-0212-4978-97d8-90420cca8650 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:43.246863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Pushing event updated for resources: {'Port': ['ID=982ed1f8-4257-4da4-9de6-458b5ec1994d,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:43.264447 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.370s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:43.307783 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:43.313069 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:43.364417 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d47c3bad-ed32-4140-81d2-3e3fec373c23 req-6f579575-b4af-485a-8b03-0896ede1f1ff tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:43.365942 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d47c3bad-ed32-4140-81d2-3e3fec373c23 req-6f579575-b4af-485a-8b03-0896ede1f1ff tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 931 time: 0.3279240 May 31 11:54:43.425107 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-46519af4-1915-44cd-a5e1-32b715e04832 req-f56c8c6a-3ac2-41fe-b142-97782f3aaaf4 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2093999 May 31 11:54:43.446596 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Attempting to bind port 982ed1f8-4257-4da4-9de6-458b5ec1994d on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:43.451291 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Attempting to bind port 982ed1f8-4257-4da4-9de6-458b5ec1994d by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'f3279336-e17b-42d0-9e25-1cefbd5c68b4', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 392, 'network_id': '02ae88e7-d3df-4dfe-bee0-364625c3e776'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:43.451291 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Attempting to bind port 982ed1f8-4257-4da4-9de6-458b5ec1994d on network 02ae88e7-d3df-4dfe-bee0-364625c3e776 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:43.468049 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Port 982ed1f8-4257-4da4-9de6-458b5ec1994d on network 02ae88e7-d3df-4dfe-bee0-364625c3e776 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:54:43.468244 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Attempting to bind port 982ed1f8-4257-4da4-9de6-458b5ec1994d on network 02ae88e7-d3df-4dfe-bee0-364625c3e776 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:43.487450 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:43.488127 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Checking segment: {'id': 'f3279336-e17b-42d0-9e25-1cefbd5c68b4', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 392, 'network_id': '02ae88e7-d3df-4dfe-bee0-364625c3e776'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:43.488860 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Bound using segment: {'id': 'f3279336-e17b-42d0-9e25-1cefbd5c68b4', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 392, 'network_id': '02ae88e7-d3df-4dfe-bee0-364625c3e776'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:43.489683 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Bound port: 982ed1f8-4257-4da4-9de6-458b5ec1994d, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f3279336-e17b-42d0-9e25-1cefbd5c68b4', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 392, 'network_id': '02ae88e7-d3df-4dfe-bee0-364625c3e776'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:43.490240 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:43.515857 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=b808043f-df25-4425-a3f4-6ee4d3a2150f,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:43.612898 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ae4fb2c2-77f4-42a8-8b8b-c8425c843ec0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:43.633996 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d47c3bad-ed32-4140-81d2-3e3fec373c23 req-857504b4-6cda-4b37-b854-86b6a78701e5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2423754 May 31 11:54:43.697016 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:43.708660 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.518s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:43.740930 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Allocated IP 172.27.1.172 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/e86adcce-af56-4a3c-8c81-4014ee2a0b72) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:43.746686 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:40Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:43.764445 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-56243bfd-de9b-4b6c-b1d5-3057a9888c43 None None] The sync data for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi is: [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:40Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:54:43.786073 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:42Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:54:43.794422 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9b824db3-6d43-4190-a330-0e18b8d15783 None None] The sync data for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi is: [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:42Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:54:43.906293 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ae4fb2c2-77f4-42a8-8b8b-c8425c843ec0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=70f30481-92c8-4fcb-bd3a-f973cd135e2a,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:43.909315 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ae4fb2c2-77f4-42a8-8b8b-c8425c843ec0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.212s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:44.039600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.070319 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id c4a233f2-9446-42a7-964d-1dc2496c938a {{(pid=99039) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:54:44.143254 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.185780 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.208021 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Transition to ACTIVE for port object c4a233f2-9446-42a7-964d-1dc2496c938a will not be triggered until provisioned by entity L2. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:44.222518 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:44.234404 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.276510 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] For port 982ed1f8-4257-4da4-9de6-458b5ec1994d, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:44.287055 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.296822 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:44.305794 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.352506 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] For port 982ed1f8-4257-4da4-9de6-458b5ec1994d, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=982ed1f8-4257-4da4-9de6-458b5ec1994d,segment=NetworkSegment(f3279336-e17b-42d0-9e25-1cefbd5c68b4),segment_id=f3279336-e17b-42d0-9e25-1cefbd5c68b4)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:44.358494 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ae4fb2c2-77f4-42a8-8b8b-c8425c843ec0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.362167 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-ae4fb2c2-77f4-42a8-8b8b-c8425c843ec0 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/70f30481-92c8-4fcb-bd3a-f973cd135e2a HTTP/1.1" status: 204 len: 168 time: 2.0343897 May 31 11:54:44.377934 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-d09d0082-ad24-44c6-a3b2-5f10863b4a9b tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:44.381150 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-d09d0082-ad24-44c6-a3b2-5f10863b4a9b tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 819 time: 0.7749643 May 31 11:54:44.395664 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'Port': ['ID=4f8bed93-6c8d-4aaf-ac82-20f075c5d327,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:44.398855 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:44.408789 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-0f96904a-0aab-4917-9701-f044bfeb5083 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/quotas/c94978b38250481095d2824adfb16ed4 HTTP/1.1" status: 200 len: 375 time: 0.0151143 May 31 11:54:44.475860 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.486189 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.489541 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:44.524388 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Pushing event updated for resources: {'Port': ['ID=c4a233f2-9446-42a7-964d-1dc2496c938a,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:44.536502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.243s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:44.540261 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:44.583028 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:44.583803 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.591326 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] 10.222.0.47 "PUT /v2.0/ports/982ed1f8-4257-4da4-9de6-458b5ec1994d HTTP/1.1" status: 200 len: 1093 time: 4.7749994 May 31 11:54:44.604948 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:44.606323 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 5.4649701 May 31 11:54:44.618666 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'd1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4', 'tag': '982ed1f8-4257-4da4-9de6-458b5ec1994d'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:54:44.909060 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Pushing event updated for resources: {'Port': ['ID=982ed1f8-4257-4da4-9de6-458b5ec1994d,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:44.944653 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:44.953406 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:44.964854 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:44.968611 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'Port': ['ID=e86adcce-af56-4a3c-8c81-4014ee2a0b72,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:44.980891 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Attempting to bind port c4a233f2-9446-42a7-964d-1dc2496c938a on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:44.981249 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Attempting to bind port c4a233f2-9446-42a7-964d-1dc2496c938a by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:44.981505 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.492s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:44.981665 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Attempting to bind port c4a233f2-9446-42a7-964d-1dc2496c938a on network 709626b6-807b-4256-9031-1578948f1fe8 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:44.995704 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.455s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:45.010665 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-ef776618-1fa4-4341-8680-027dc107faf0 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&fields=id HTTP/1.1" status: 200 len: 255 time: 0.5914619 May 31 11:54:45.052587 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Port c4a233f2-9446-42a7-964d-1dc2496c938a on network 709626b6-807b-4256-9031-1578948f1fe8 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:54:45.052902 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Attempting to bind port c4a233f2-9446-42a7-964d-1dc2496c938a on network 709626b6-807b-4256-9031-1578948f1fe8 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:45.103964 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:45.105377 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Checking segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:45.106149 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Bound using segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:45.106794 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Bound port: c4a233f2-9446-42a7-964d-1dc2496c938a, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'}}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:45.107146 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:45.158739 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-a5515b52-8d00-479e-8cac-33b58e100410" -d '{"events": [{"name": "network-changed", "server_uuid": "d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4", "tag": "982ed1f8-4257-4da4-9de6-458b5ec1994d"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:54:45.171620 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-340d044c-5800-422f-83c2-03ee28b04475 req-1d0da1cb-9640-442e-a42f-d509afc58e36 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:45.173184 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-1d0da1cb-9640-442e-a42f-d509afc58e36 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=3b87c0540dea485d8ec60a8628fced11&device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 1096 time: 0.2730517 May 31 11:54:45.241983 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:45.318968 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6337a5eb-7679-4109-95d4-f1cc96ff0e7c req-39fb59c9-0bae-46df-9575-8f7436c61c35 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:45.327156 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6337a5eb-7679-4109-95d4-f1cc96ff0e7c req-39fb59c9-0bae-46df-9575-8f7436c61c35 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 943 time: 0.5380235 May 31 11:54:45.360766 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-59b25359-191c-4617-8cd2-19f83a778319 service neutron] 10.222.0.48 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3412213 May 31 11:54:45.381284 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-ccdb7737-0f43-49ba-9da0-68e8cbc8ab62 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=982ed1f8-4257-4da4-9de6-458b5ec1994d HTTP/1.1" status: 200 len: 217 time: 0.1978052 May 31 11:54:45.429099 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.47', 49565) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:45.458786 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57874) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:45.506829 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6337a5eb-7679-4109-95d4-f1cc96ff0e7c req-bf99aea2-04c5-4179-a346-52ec24362fbc tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1785660 May 31 11:54:45.537059 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] For port c4a233f2-9446-42a7-964d-1dc2496c938a, host hv-cinder-79123, cleared binding levels {{(pid=99039) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:45.637185 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b5af8b49-c5f1-494f-aa5a-3d080497d163 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:45.830884 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:45.830884 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:45.830884 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-340d044c-5800-422f-83c2-03ee28b04475 req-cceacbe7-72ee-4e8f-bec0-102ad5ac36d8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:45.830884 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-cceacbe7-72ee-4e8f-bec0-102ad5ac36d8 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=94658dd6-8c5b-487d-9286-e8ccb8e389f4 HTTP/1.1" status: 200 len: 852 time: 0.3425062 May 31 11:54:45.831451 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:45.831758 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Usage tracker for resource:security_group_rule and tenant:e1063f1164e2491a80c040fbec9c021a is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:45.831758 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:45.832150 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] For port c4a233f2-9446-42a7-964d-1dc2496c938a, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c4a233f2-9446-42a7-964d-1dc2496c938a,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99039) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:45.832224 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-c650f320-f1c1-4d2b-91d7-2a8462eff183 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=02ae88e7-d3df-4dfe-bee0-364625c3e776&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0588596 May 31 11:54:45.832224 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:45.862767 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1032ab06-097d-470f-8987-43238090f951 service neutron] 10.222.0.48 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4802127 May 31 11:54:46.001559 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f21012f6-9cfa-442f-b326-42af02da004c req-70b78575-3d05-47cd-b81c-374db5ee7be4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:46.001559 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-70b78575-3d05-47cd-b81c-374db5ee7be4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 819 time: 0.5053661 May 31 11:54:46.001559 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Request body: {'port': {'device_id': '66c4d526-5295-45db-9e0b-516b82570603', 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999', 'admin_state_up': True, 'tenant_id': 'c94978b38250481095d2824adfb16ed4'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:46.057952 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-e892080c-1986-40b1-a870-3ab01dd2e4cb service neutron] 10.222.0.47 "GET /v2.0/networks/02ae88e7-d3df-4dfe-bee0-364625c3e776?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2268584 May 31 11:54:46.196311 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.240778 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-340d044c-5800-422f-83c2-03ee28b04475 req-83e490e5-da03-4b78-bdbf-75eefcc6f839 service neutron] 10.222.0.47 "GET /v2.0/networks/02ae88e7-d3df-4dfe-bee0-364625c3e776?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1738634 May 31 11:54:46.243297 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4a75d56-c5a3-45fd-be8f-a1b3c28dedb2 req-d3c255c4-b0eb-4fd5-85e3-6f6ed8472ff0 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:46.245614 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e4a75d56-c5a3-45fd-be8f-a1b3c28dedb2 req-d3c255c4-b0eb-4fd5-85e3-6f6ed8472ff0 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 943 time: 0.4834290 May 31 11:54:46.359758 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:46.365310 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e4a75d56-c5a3-45fd-be8f-a1b3c28dedb2 req-e274a9ad-b799-487d-8920-eef57b80119f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1084919 May 31 11:54:46.375514 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:46.399225 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.448814 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port b5af8b49-c5f1-494f-aa5a-3d080497d163 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:46.451696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.462306 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port b5af8b49-c5f1-494f-aa5a-3d080497d163 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:46.462306 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.462610 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:46.491727 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.514715 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c9baca86-a559-4b8f-a630-df1dc908f80d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:46.549390 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.599519 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8d5fc247-8e84-4141-9f84-e46d07178f7d,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:46.607435 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:46.617847 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.620020 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] 10.222.0.47 "PUT /v2.0/ports/c4a233f2-9446-42a7-964d-1dc2496c938a HTTP/1.1" status: 200 len: 1093 time: 3.9359791 May 31 11:54:46.620942 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=e1063f1164e2491a80c040fbec9c021a&name=default HTTP/1.1" status: 200 len: 2979 time: 1.1532164 May 31 11:54:46.621827 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 57896) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:46.621975 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bb55be8b-198d-40e7-849a-8543fbf35000', 'tag': 'c4a233f2-9446-42a7-964d-1dc2496c938a'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:54:46.622856 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:46.701375 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.729333 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Made reservation on behalf of c94978b38250481095d2824adfb16ed4 for: {'port': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:46.732774 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.745007 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c64448ac-d6fe-46e9-b30e-5aa58dd935d9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.833894 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c64448ac-d6fe-46e9-b30e-5aa58dd935d9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.834481 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port b5af8b49-c5f1-494f-aa5a-3d080497d163, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b5af8b49-c5f1-494f-aa5a-3d080497d163,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:46.887184 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.907949 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-4cc1a663-0520-4c48-874c-07256223e3ac" -d '{"events": [{"name": "network-changed", "server_uuid": "bb55be8b-198d-40e7-849a-8543fbf35000", "tag": "c4a233f2-9446-42a7-964d-1dc2496c938a"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:54:46.912774 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-58ac28fe-032a-464c-a094-7026208f389a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:security_group {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:46.937639 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:54:45 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-2b22a42c-2718-4329-905d-899289c68607 x-openstack-request-id: req-2b22a42c-2718-4329-905d-899289c68607 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:54:46.937926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4", "tag": "982ed1f8-4257-4da4-9de6-458b5ec1994d", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:54:46.938215 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-2b22a42c-2718-4329-905d-899289c68607 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:54:46.938690 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'd1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4', 'tag': '982ed1f8-4257-4da4-9de6-458b5ec1994d', 'status': 'completed', 'code': 200} May 31 11:54:46.978766 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:46.980111 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 502e1922-1254-4eed-bdc8-4c83694840b3 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:46.988165 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Pushing event updated for resources: {'Port': ['ID=c4a233f2-9446-42a7-964d-1dc2496c938a,revision_number=3']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:46.995365 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.535s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:46.997174 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.416s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:47.057734 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-48b41d82-ed23-4326-9825-8dfe6a6a5b97 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:47.064404 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a5515b52-8d00-479e-8cac-33b58e100410 req-1ed0bd4c-e69a-48ac-806c-1d502c29f6d7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:47.066280 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a5515b52-8d00-479e-8cac-33b58e100410 req-1ed0bd4c-e69a-48ac-806c-1d502c29f6d7 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=3b87c0540dea485d8ec60a8628fced11&device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 1096 time: 0.1017721 May 31 11:54:47.099099 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'Port': ['ID=e86adcce-af56-4a3c-8c81-4014ee2a0b72,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:47.114638 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:47.126268 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-685d21bd-f2f2-4b4b-9589-bb2f37c8b62d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:47.143131 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-685d21bd-f2f2-4b4b-9589-bb2f37c8b62d service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 1096 time: 0.3334346 May 31 11:54:47.259836 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-58ac28fe-032a-464c-a094-7026208f389a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:security_group {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:47.269119 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-58ac28fe-032a-464c-a094-7026208f389a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-securitygroup--853597934&fields=id&tenant_id=b866e6eab1b04406a7280f91698983ce HTTP/1.1" status: 200 len: 267 time: 0.8117025 May 31 11:54:47.280745 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-2a19710f-8f7a-480c-8f27-7396b76e5b73 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=c4a233f2-9446-42a7-964d-1dc2496c938a HTTP/1.1" status: 200 len: 217 time: 0.1444976 May 31 11:54:47.300177 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c64448ac-d6fe-46e9-b30e-5aa58dd935d9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:47.314213 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=b5af8b49-c5f1-494f-aa5a-3d080497d163,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:47.335421 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.278s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:47.407664 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 502e1922-1254-4eed-bdc8-4c83694840b3 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:47.413644 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 502e1922-1254-4eed-bdc8-4c83694840b3 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:47.414323 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:47.417965 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:47.425459 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:47.426677 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 5.3334174 May 31 11:54:47.453820 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c64448ac-d6fe-46e9-b30e-5aa58dd935d9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8d5fc247-8e84-4141-9f84-e46d07178f7d,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:47.457114 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c64448ac-d6fe-46e9-b30e-5aa58dd935d9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:47.465310 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Allocated IP 172.20.0.12 (c36bb4a9-2822-49b1-b291-69604a881999/6312c1ab-b503-4bcc-af02-8274dbf2679c/f18f632c-d53b-4423-93a7-f93c3e8e0778) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:47.492004 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c64448ac-d6fe-46e9-b30e-5aa58dd935d9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:47.493586 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c64448ac-d6fe-46e9-b30e-5aa58dd935d9 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/8d5fc247-8e84-4141-9f84-e46d07178f7d HTTP/1.1" status: 204 len: 168 time: 0.8686678 May 31 11:54:47.529994 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a5515b52-8d00-479e-8cac-33b58e100410 req-4aab4138-587e-4347-abc9-6dcc4cc7ea45 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:47.531488 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a5515b52-8d00-479e-8cac-33b58e100410 req-4aab4138-587e-4347-abc9-6dcc4cc7ea45 service neutron] 10.222.0.47 "GET /v2.0/networks?id=02ae88e7-d3df-4dfe-bee0-364625c3e776 HTTP/1.1" status: 200 len: 913 time: 0.4572508 May 31 11:54:47.544455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-910b74ab-3b6c-4002-a1d0-f0c5034de6ed req-fba4a805-e97c-43e3-b65c-e4d93c2587e9 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:47.546002 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-910b74ab-3b6c-4002-a1d0-f0c5034de6ed req-fba4a805-e97c-43e3-b65c-e4d93c2587e9 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 943 time: 0.2556446 May 31 11:54:47.552979 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-26ce7298-d4f8-4d3c-8062-4aea0e9f9b9a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:47.554352 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-26ce7298-d4f8-4d3c-8062-4aea0e9f9b9a service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 200 len: 855 time: 0.2499006 May 31 11:54:47.649934 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-1fb8cede-9fb6-407c-85d8-83792eded3b2 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=709626b6-807b-4256-9031-1578948f1fe8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0811169 May 31 11:54:47.656703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-fae19c84-2666-432e-b75a-8fc3ebe42cb2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:47.664495 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-fae19c84-2666-432e-b75a-8fc3ebe42cb2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 811 time: 0.3943567 May 31 11:54:47.752348 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a5515b52-8d00-479e-8cac-33b58e100410 req-79f22956-3dce-4278-8d6b-7e9cf1db8efe service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=982ed1f8-4257-4da4-9de6-458b5ec1994d HTTP/1.1" status: 200 len: 217 time: 0.2051997 May 31 11:54:47.790320 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-910b74ab-3b6c-4002-a1d0-f0c5034de6ed req-c620c441-375e-4caf-886f-751970146a94 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1690462 May 31 11:54:47.803295 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:47.848667 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 502e1922-1254-4eed-bdc8-4c83694840b3, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=502e1922-1254-4eed-bdc8-4c83694840b3,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:47.859219 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-ef9b80eb-b093-4f7d-b25a-8f984d5cafe2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/quotas/b866e6eab1b04406a7280f91698983ce HTTP/1.1" status: 200 len: 375 time: 0.1935892 May 31 11:54:47.882458 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7f7d970b-c1e2-40d8-814c-ddbb0c0faa7b req-2a8feb68-8a5c-4c5c-95fa-55b7a4144b58 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 210 time: 0.0432448 May 31 11:54:47.967895 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:47.969007 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b808043f-df25-4425-a3f4-6ee4d3a2150f up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:48.003092 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:48.046384 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-2a1df710-7515-4558-8ff3-61893a0b5ab7 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3729441 May 31 11:54:48.050489 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:48.076856 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-f74c7e26-fa54-481c-a899-8f1221276cd3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&fields=id HTTP/1.1" status: 200 len: 348 time: 0.2057610 May 31 11:54:48.117033 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=502e1922-1254-4eed-bdc8-4c83694840b3,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:48.129997 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:48.130616 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port b808043f-df25-4425-a3f4-6ee4d3a2150f triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:48.131153 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:48.176236 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:48.180673 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:54:46 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-9b77288c-3ef2-4ae2-a899-54ddf47a8fb9 x-openstack-request-id: req-9b77288c-3ef2-4ae2-a899-54ddf47a8fb9 {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:54:48.180673 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "bb55be8b-198d-40e7-849a-8543fbf35000", "tag": "c4a233f2-9446-42a7-964d-1dc2496c938a", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:54:48.180673 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-9b77288c-3ef2-4ae2-a899-54ddf47a8fb9 {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:54:48.180673 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bb55be8b-198d-40e7-849a-8543fbf35000', 'tag': 'c4a233f2-9446-42a7-964d-1dc2496c938a', 'status': 'completed', 'code': 200} May 31 11:54:48.198591 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-00e17aa3-7031-424c-902a-3ee4421a1ece admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:48.286077 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Network c36bb4a9-2822-49b1-b291-69604a881999 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:48.300487 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:48.306873 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:48.312266 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:48.312885 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:48.334626 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Pushing event updated for resources: {'Port': ['ID=f18f632c-d53b-4423-93a7-f93c3e8e0778,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:48.340920 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-4d1bcab5-7f9b-4193-b363-3598a53d2fb2 service neutron] 10.222.0.48 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2471938 May 31 11:54:48.342420 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:48.472778 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b5af8b49-c5f1-494f-aa5a-3d080497d163 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:48.477421 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 2.5230834 May 31 11:54:48.516383 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7cac1ed-5cf1-48a1-a2e4-14b0c8ce1773 req-5e9636a6-af56-4f7f-997b-7b5f60b2a682 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:48.525129 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a7cac1ed-5cf1-48a1-a2e4-14b0c8ce1773 req-5e9636a6-af56-4f7f-997b-7b5f60b2a682 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 943 time: 0.2386060 May 31 11:54:48.567079 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port b5af8b49-c5f1-494f-aa5a-3d080497d163 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:48.567458 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:48.580730 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-a7cac1ed-5cf1-48a1-a2e4-14b0c8ce1773 req-b0716935-b4fd-4f55-9ee7-43cb4ffadd94 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:48.600648 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-13bd5092-da7a-496b-a74e-ff67050de77f service neutron] 10.222.0.48 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2520103 May 31 11:54:48.676310 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-a7cac1ed-5cf1-48a1-a2e4-14b0c8ce1773 req-b0716935-b4fd-4f55-9ee7-43cb4ffadd94 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:48.677421 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a7cac1ed-5cf1-48a1-a2e4-14b0c8ce1773 req-b0716935-b4fd-4f55-9ee7-43cb4ffadd94 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1515169 May 31 11:54:48.718052 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 502e1922-1254-4eed-bdc8-4c83694840b3 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:48.788268 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 502e1922-1254-4eed-bdc8-4c83694840b3 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:48.788879 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:48.826135 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-14986fb5-7bb7-4cc2-a359-86d26cf2051f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-securitygroup--569520699&fields=id&tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d HTTP/1.1" status: 200 len: 267 time: 0.0461495 May 31 11:54:49.017541 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-b1f0537b-afee-4aba-8851-643e4e8874a2 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:49.019339 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-b1f0537b-afee-4aba-8851-643e4e8874a2 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 816 time: 0.1939130 May 31 11:54:49.051798 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-1d0c8992-9ff7-49a0-9d42-8dd3ef7e5221 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/quotas/0e7f0a333ee54b31bcd5c2290c3e284d HTTP/1.1" status: 200 len: 375 time: 0.0247605 May 31 11:54:49.120544 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de76f324-8a73-4191-bdda-db00aa71658b req-e472d2df-eefe-40af-b07b-155315a62b95 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:49.127881 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-de76f324-8a73-4191-bdda-db00aa71658b req-e472d2df-eefe-40af-b07b-155315a62b95 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 943 time: 0.1166959 May 31 11:54:49.174394 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-d431345e-7c16-47df-835e-6c4ccb9f13bd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Synchronizing usage tracker for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:49.182558 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-de76f324-8a73-4191-bdda-db00aa71658b req-0b9f80d8-ca06-428b-9288-a062a44556c7 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0533328 May 31 11:54:49.268646 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2b16c9a7-1dd0-4bc8-9282-dba547180905 req-594ba35b-4e6c-49af-b537-14dd1d9e435e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 210 time: 0.0450783 May 31 11:54:49.290640 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-d431345e-7c16-47df-835e-6c4ccb9f13bd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Unset dirty status for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:49.292009 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-d431345e-7c16-47df-835e-6c4ccb9f13bd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&fields=id HTTP/1.1" status: 200 len: 348 time: 0.2280104 May 31 11:54:49.406609 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-2e8c3032-1a50-4ff2-a755-5751234b628b req-30b37b5b-5203-4528-aa7e-750d9e862e5b tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Synchronizing usage tracker for tenant:c94978b38250481095d2824adfb16ed4 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:49.444604 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-91dd22b5-b333-4de9-9b4f-d2c0ceb9b976 service neutron] 10.222.0.48 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1342506 May 31 11:54:49.528012 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-2e8c3032-1a50-4ff2-a755-5751234b628b req-30b37b5b-5203-4528-aa7e-750d9e862e5b tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Unset dirty status for tenant:c94978b38250481095d2824adfb16ed4 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:49.538358 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2e8c3032-1a50-4ff2-a755-5751234b628b req-30b37b5b-5203-4528-aa7e-750d9e862e5b tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 210 time: 0.1710379 May 31 11:54:49.578589 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-b1f52082-167d-4023-a5f5-1b476f300c87 service neutron] 10.222.0.48 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1314631 May 31 11:54:49.624705 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 982ed1f8-4257-4da4-9de6-458b5ec1994d details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:54:49.782712 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 982ed1f8-4257-4da4-9de6-458b5ec1994d, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=982ed1f8-4257-4da4-9de6-458b5ec1994d,segment=NetworkSegment(f3279336-e17b-42d0-9e25-1cefbd5c68b4),segment_id=f3279336-e17b-42d0-9e25-1cefbd5c68b4)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:49.802632 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '982ed1f8-4257-4da4-9de6-458b5ec1994d', 'network_id': '02ae88e7-d3df-4dfe-bee0-364625c3e776', 'port_id': '982ed1f8-4257-4da4-9de6-458b5ec1994d', 'mac_address': 'fa:16:3e:3a:a5:26', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 392, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '94658dd6-8c5b-487d-9286-e8ccb8e389f4', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:54:49.870882 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:49.879032 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbf55a0a-598b-4a36-b5b2-3bb79d9babdf req-f257e9dc-5fb9-483c-9aa7-2bc74b014110 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:49.887067 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fbf55a0a-598b-4a36-b5b2-3bb79d9babdf req-f257e9dc-5fb9-483c-9aa7-2bc74b014110 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 943 time: 0.0948710 May 31 11:54:49.909872 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 982ed1f8-4257-4da4-9de6-458b5ec1994d {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:54:49.959491 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 982ed1f8-4257-4da4-9de6-458b5ec1994d, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=982ed1f8-4257-4da4-9de6-458b5ec1994d,segment=NetworkSegment(f3279336-e17b-42d0-9e25-1cefbd5c68b4),segment_id=f3279336-e17b-42d0-9e25-1cefbd5c68b4)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:49.980689 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fbf55a0a-598b-4a36-b5b2-3bb79d9babdf req-5d2d530e-7700-4e07-81c0-6a1eaf59ea5d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0920150 May 31 11:54:49.985249 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7e90dbae-5afc-4319-a638-ba9aee72808a req-7d7400b1-6bfc-4555-908f-bef6202cd58b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 210 time: 0.0903690 May 31 11:54:50.035722 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:50.038055 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-87f8168a-c611-4d19-b7f7-39aedada206f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:50.111984 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=982ed1f8-4257-4da4-9de6-458b5ec1994d,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:50.114960 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:50.382022 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ad15e23-2fb6-440e-89c1-cf579ad9fde4 req-4618d29e-d93a-4eb8-8fe9-a6fc04f6ae17 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:50.383765 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0ad15e23-2fb6-440e-89c1-cf579ad9fde4 req-4618d29e-d93a-4eb8-8fe9-a6fc04f6ae17 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 944 time: 0.0905280 May 31 11:54:50.435781 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-959c4280-e1e6-4a60-afe4-ff31ba7bc136 req-a3a04ff0-c8d3-419b-8f67-c20ec0cda93b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:50.444189 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0ad15e23-2fb6-440e-89c1-cf579ad9fde4 req-231c8d97-657e-440d-9f32-1652a82b7220 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0545008 May 31 11:54:50.562087 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-959c4280-e1e6-4a60-afe4-ff31ba7bc136 req-a3a04ff0-c8d3-419b-8f67-c20ec0cda93b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:50.563268 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-959c4280-e1e6-4a60-afe4-ff31ba7bc136 req-a3a04ff0-c8d3-419b-8f67-c20ec0cda93b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 210 time: 0.1699386 May 31 11:54:50.711265 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a5515b52-8d00-479e-8cac-33b58e100410 req-a549b8c3-286d-484f-bff3-20d3d47e7cd6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:50.713042 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a5515b52-8d00-479e-8cac-33b58e100410 req-a549b8c3-286d-484f-bff3-20d3d47e7cd6 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=94658dd6-8c5b-487d-9286-e8ccb8e389f4 HTTP/1.1" status: 200 len: 852 time: 0.0802572 May 31 11:54:50.720122 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Request body: {'port': {'device_id': '66c4d526-5295-45db-9e0b-516b82570603', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:50.765623 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-309909b4-468b-4631-af0f-6b81001fd85c req-2bea6eba-b003-445c-a502-aba47464b9c6 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 210 time: 0.1307096 May 31 11:54:50.794701 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a5515b52-8d00-479e-8cac-33b58e100410 req-2b1c809f-5e80-4a7f-9326-5f8c9dbc7437 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=02ae88e7-d3df-4dfe-bee0-364625c3e776&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0629971 May 31 11:54:50.899086 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['982ed1f8-4257-4da4-9de6-458b5ec1994d'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:54:50.935562 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a5515b52-8d00-479e-8cac-33b58e100410 req-c2c6a52d-6c87-4c0d-b8df-1ae5231b26e2 service neutron] 10.222.0.47 "GET /v2.0/networks/02ae88e7-d3df-4dfe-bee0-364625c3e776?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1388938 May 31 11:54:50.943203 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-bfa1281f-73cf-46f2-9120-80f45622f6a5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:51.000742 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b75b3c0-f97e-4aa5-b643-f15dfb938d48 req-962e9edd-3703-4f54-a5af-5f864a5a3f25 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3225775 May 31 11:54:51.061531 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:51.179548 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a5515b52-8d00-479e-8cac-33b58e100410 req-5afd253b-0626-45a7-a749-add32f8278d4 service neutron] 10.222.0.47 "GET /v2.0/networks/02ae88e7-d3df-4dfe-bee0-364625c3e776?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2231553 May 31 11:54:51.220778 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75cf74f8-0d6c-4539-8220-a1df6ad38668 req-6e06aff1-8c4f-46ca-8b0e-3402e2dee213 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 210 time: 0.0971534 May 31 11:54:51.403558 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-292af2cc-4bca-44df-b313-94709ce0e6eb req-fb0e6131-729e-4129-a3c8-d5af41265115 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:51.410873 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-292af2cc-4bca-44df-b313-94709ce0e6eb req-fb0e6131-729e-4129-a3c8-d5af41265115 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 943 time: 0.2278643 May 31 11:54:51.442605 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4cc1a663-0520-4c48-874c-07256223e3ac req-307ed831-ae0c-4c33-b3bb-b44b736ea073 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:51.444044 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cc1a663-0520-4c48-874c-07256223e3ac req-307ed831-ae0c-4c33-b3bb-b44b736ea073 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 1096 time: 0.2002532 May 31 11:54:51.582120 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id f18f632c-d53b-4423-93a7-f93c3e8e0778 {{(pid=99039) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:54:51.590743 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-292af2cc-4bca-44df-b313-94709ce0e6eb req-70f8c8cd-bcc5-4780-8f40-befcbc2f4043 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1788633 May 31 11:54:51.593597 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:51.660212 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Transition to ACTIVE for port object f18f632c-d53b-4423-93a7-f93c3e8e0778 will not be triggered until provisioned by entity L2. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:51.663554 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4cc1a663-0520-4c48-874c-07256223e3ac req-ef6d6fb3-5e52-4e42-86f6-e729ddcbd72b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:51.665329 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cc1a663-0520-4c48-874c-07256223e3ac req-ef6d6fb3-5e52-4e42-86f6-e729ddcbd72b service neutron] 10.222.0.47 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 916 time: 0.2126946 May 31 11:54:51.709177 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa21c916-dcd3-4653-a44f-fdbe7fd75f06 req-f213a3d3-11ae-456e-91ca-0cfe63036de2 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:51.711058 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-aa21c916-dcd3-4653-a44f-fdbe7fd75f06 req-f213a3d3-11ae-456e-91ca-0cfe63036de2 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 944 time: 0.1144893 May 31 11:54:51.748654 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cc1a663-0520-4c48-874c-07256223e3ac req-3bd36799-cbe1-4a53-a99a-17c1c2ad8ac4 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=c4a233f2-9446-42a7-964d-1dc2496c938a HTTP/1.1" status: 200 len: 217 time: 0.0760117 May 31 11:54:51.751622 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:51.762209 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-49a85a89-6e55-4ea7-9ca2-434aaca35f3c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:51.807879 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-aa21c916-dcd3-4653-a44f-fdbe7fd75f06 req-5b4f7069-3120-4e6c-9d17-e5a71dd2c7ae tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0899963 May 31 11:54:51.911341 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9a2267b2-8ccf-46af-8dd1-e91760432ead req-7ff2a1e6-9b03-46d2-8417-f6b94e955ae0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 210 time: 0.1616838 May 31 11:54:51.943365 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4cc1a663-0520-4c48-874c-07256223e3ac req-bdda1bf9-640f-46d8-8707-9e25e8c3cd4b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:51.949133 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cc1a663-0520-4c48-874c-07256223e3ac req-bdda1bf9-640f-46d8-8707-9e25e8c3cd4b service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 200 len: 855 time: 0.1627915 May 31 11:54:51.995819 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cc1a663-0520-4c48-874c-07256223e3ac req-fed176cc-a84b-458b-8571-ed615668dfd6 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=709626b6-807b-4256-9031-1578948f1fe8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0434439 May 31 11:54:52.111696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Pushing event updated for resources: {'Port': ['ID=f18f632c-d53b-4423-93a7-f93c3e8e0778,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:52.126433 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.364s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:52.130512 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c48c4508-6389-4760-88b2-21ad06383d57 req-ff1e7607-62dc-4dc9-97a3-fd2dfa34d5e6 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 210 time: 0.0690882 May 31 11:54:52.132725 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Attempting to bind port f18f632c-d53b-4423-93a7-f93c3e8e0778 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:52.133055 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Attempting to bind port f18f632c-d53b-4423-93a7-f93c3e8e0778 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:52.133785 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Attempting to bind port f18f632c-d53b-4423-93a7-f93c3e8e0778 on network c36bb4a9-2822-49b1-b291-69604a881999 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:52.138874 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Port f18f632c-d53b-4423-93a7-f93c3e8e0778 on network c36bb4a9-2822-49b1-b291-69604a881999 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:54:52.139188 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Attempting to bind port f18f632c-d53b-4423-93a7-f93c3e8e0778 on network c36bb4a9-2822-49b1-b291-69604a881999 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:52.144462 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:52.148813 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Checking segment: {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:52.148813 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Bound using segment: {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:52.148813 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Bound port: f18f632c-d53b-4423-93a7-f93c3e8e0778, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'}}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:52.148813 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:52.178140 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cc1a663-0520-4c48-874c-07256223e3ac req-78dc707c-b413-49f8-b78d-0c7a59ec97a2 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1715937 May 31 11:54:52.358764 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cc1a663-0520-4c48-874c-07256223e3ac req-a3f1ea1e-ae4b-4766-8dba-d0958584f435 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1669729 May 31 11:54:52.449910 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] For port f18f632c-d53b-4423-93a7-f93c3e8e0778, host hv-cinder-79123, cleared binding levels {{(pid=99039) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:52.471291 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f21abc9-411b-4789-932c-40f7e8e1e1cf req-985cf5fc-f84c-48bd-8e67-db8255dfcaff tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 210 time: 0.0578606 May 31 11:54:52.484752 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-aa61babe-6096-4a66-bcfb-99c7ae3236ac tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:52.486588 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-aa61babe-6096-4a66-bcfb-99c7ae3236ac tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 811 time: 0.2364781 May 31 11:54:52.491904 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] For port f18f632c-d53b-4423-93a7-f93c3e8e0778, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99039) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:52.567054 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:52.576928 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-ac0bd70d-48ab-4944-98cb-76bc8f952ced tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=b866e6eab1b04406a7280f91698983ce HTTP/1.1" status: 200 len: 379 time: 0.0705416 May 31 11:54:52.579370 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:52.645663 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'port': {'device_id': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'admin_state_up': True, 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'security_groups': ['4555883c-aba9-4fc6-894a-6ddaa27e8399']}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:52.678686 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:52.679321 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:52.680703 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] 10.222.0.47 "PUT /v2.0/ports/f18f632c-d53b-4423-93a7-f93c3e8e0778 HTTP/1.1" status: 200 len: 1094 time: 2.0037673 May 31 11:54:52.728663 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '66c4d526-5295-45db-9e0b-516b82570603', 'tag': 'f18f632c-d53b-4423-93a7-f93c3e8e0778'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:54:52.730951 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-dc983b20-7f09-4c60-83ce-1bd98bb94da0" -d '{"events": [{"name": "network-changed", "server_uuid": "66c4d526-5295-45db-9e0b-516b82570603", "tag": "f18f632c-d53b-4423-93a7-f93c3e8e0778"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:54:52.822996 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:54:52 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-a7be9ddd-56ca-4e54-84f8-833d5cd3e028 x-openstack-request-id: req-a7be9ddd-56ca-4e54-84f8-833d5cd3e028 {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:54:52.823305 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "66c4d526-5295-45db-9e0b-516b82570603", "tag": "f18f632c-d53b-4423-93a7-f93c3e8e0778", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:54:52.823569 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-a7be9ddd-56ca-4e54-84f8-833d5cd3e028 {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:54:52.824007 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '66c4d526-5295-45db-9e0b-516b82570603', 'tag': 'f18f632c-d53b-4423-93a7-f93c3e8e0778', 'status': 'completed', 'code': 200} May 31 11:54:52.827106 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Pushing event updated for resources: {'Port': ['ID=f18f632c-d53b-4423-93a7-f93c3e8e0778,revision_number=3']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:52.842977 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.258s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:52.968250 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:52.992394 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:53.021802 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a200b06-5d4a-4312-941b-6c1602579db4 req-08bd88f6-f2d3-40db-b219-9071a7260fc8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:53.029132 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8a200b06-5d4a-4312-941b-6c1602579db4 req-08bd88f6-f2d3-40db-b219-9071a7260fc8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 943 time: 0.2889097 May 31 11:54:53.091109 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc983b20-7f09-4c60-83ce-1bd98bb94da0 req-ae2ce529-0176-47a3-9fe3-a16bf8bffe8c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:53.094376 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-dc983b20-7f09-4c60-83ce-1bd98bb94da0 req-ae2ce529-0176-47a3-9fe3-a16bf8bffe8c service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1097 time: 0.2314761 May 31 11:54:53.098449 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Made reservation on behalf of b866e6eab1b04406a7280f91698983ce for: {'port': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:53.104499 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:53.129413 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8a200b06-5d4a-4312-941b-6c1602579db4 req-af70f45a-1cf6-4420-ba06-874f2a219cbe tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0993116 May 31 11:54:53.199382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-b0cc529a-6fe4-485e-849b-007719191f32 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:53.201178 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-b0cc529a-6fe4-485e-849b-007719191f32 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 819 time: 0.2857847 May 31 11:54:53.230766 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Request body: {'port': {'device_id': '71751849-4dff-4706-b4fe-d196e7c2e6e2', 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999', 'admin_state_up': True, 'tenant_id': 'c94978b38250481095d2824adfb16ed4'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:53.251600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c25fd776-eaf0-4cdd-94b3-4820ddc1f214 req-9b049203-8843-4381-8d2d-7c241fba0033 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:53.253317 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c25fd776-eaf0-4cdd-94b3-4820ddc1f214 req-9b049203-8843-4381-8d2d-7c241fba0033 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 944 time: 0.1939657 May 31 11:54:53.337233 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6c35f64b-af6e-4651-b91c-88735bb2f18e req-88ca74d7-da2b-48e6-85dc-174dfc99ac3e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 210 time: 0.0739536 May 31 11:54:53.351765 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:53.387397 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c25fd776-eaf0-4cdd-94b3-4820ddc1f214 req-efd11718-dee1-4fe8-95f9-50e5eebecb1d tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1229277 May 31 11:54:53.395122 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc983b20-7f09-4c60-83ce-1bd98bb94da0 req-3fe54bdc-277c-40db-ad01-7ff56ff7c576 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:53.397068 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-dc983b20-7f09-4c60-83ce-1bd98bb94da0 req-3fe54bdc-277c-40db-ad01-7ff56ff7c576 service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=c94978b38250481095d2824adfb16ed4&shared=False HTTP/1.1" status: 200 len: 920 time: 0.2821229 May 31 11:54:53.451949 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:53.461991 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:53.506589 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8dffe9fb-81f1-4d9e-b566-200a293c52a0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:53.529422 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 982ed1f8-4257-4da4-9de6-458b5ec1994d up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:53.590647 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1a673e16-1cc9-49e3-811c-4c2f5469c216 req-fa3e832e-6d28-4de4-b8e0-2f08c54196c9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 210 time: 0.0748336 May 31 11:54:53.612411 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Allocated IP 172.20.0.5 (36cce22a-502c-464d-a9ac-50f2bc247e07/1b74adba-cf2f-4db5-ad92-27ea39726767/d447eadc-67c7-4eca-8295-e5ae670ce616) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:53.612824 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Made reservation on behalf of c94978b38250481095d2824adfb16ed4 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:53.616631 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:53.660030 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.47', 49580) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:53.683388 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-bf078045-002b-4875-9130-4b6a5f3f2191 req-63f6a11e-44dc-4189-bce3-354332ae8b48 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 210 time: 0.0509226 May 31 11:54:53.687165 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 982ed1f8-4257-4da4-9de6-458b5ec1994d completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:53.692849 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 982ed1f8-4257-4da4-9de6-458b5ec1994d triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:53.693518 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:53.714811 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc983b20-7f09-4c60-83ce-1bd98bb94da0 req-3c1c260c-af5f-4419-a725-06815e9e26aa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:53.714811 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-dc983b20-7f09-4c60-83ce-1bd98bb94da0 req-3c1c260c-af5f-4419-a725-06815e9e26aa service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2789478 May 31 11:54:53.878310 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:53.880526 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Allocated IP 172.20.0.7 (c36bb4a9-2822-49b1-b291-69604a881999/6312c1ab-b503-4bcc-af02-8274dbf2679c/3670cfae-0d92-4d02-a5ba-5c373efc785e) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:53.928285 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '982ed1f8-4257-4da4-9de6-458b5ec1994d'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:54:53.998839 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 982ed1f8-4257-4da4-9de6-458b5ec1994d, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=982ed1f8-4257-4da4-9de6-458b5ec1994d,segment=NetworkSegment(f3279336-e17b-42d0-9e25-1cefbd5c68b4),segment_id=f3279336-e17b-42d0-9e25-1cefbd5c68b4)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:54.099058 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:54.099058 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-afdf8803-ba31-4995-a1cb-d50fac2bdc3f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:54.195848 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-4792b347-af69-4554-8f91-f9cb8df1c8c9" -d '{"events": [{"server_uuid": "d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4", "name": "network-vif-plugged", "status": "completed", "tag": "982ed1f8-4257-4da4-9de6-458b5ec1994d"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:54:54.199317 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=982ed1f8-4257-4da4-9de6-458b5ec1994d,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:54.199483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:54.205537 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:54.217621 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-8fc71910-652b-4185-a659-882d54470e02 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:54.217621 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-8fc71910-652b-4185-a659-882d54470e02 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Synchronizing usage tracker for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:network {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:54.258935 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f21012f6-9cfa-442f-b326-42af02da004c req-1fd801cc-3d76-4529-b235-247359834fc2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:54.260963 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-1fd801cc-3d76-4529-b235-247359834fc2 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1097 time: 0.4712162 May 31 11:54:54.378720 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-7eef5079-e380-421e-a72e-9ecbaa38cdd0 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778 HTTP/1.1" status: 200 len: 217 time: 0.1088521 May 31 11:54:54.381069 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:54.387057 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:54:54 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-9a33bfec-30d9-4d7d-8c2b-1b498dcecadf x-openstack-request-id: req-9a33bfec-30d9-4d7d-8c2b-1b498dcecadf {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:54:54.387057 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4", "name": "network-vif-plugged", "status": "completed", "tag": "982ed1f8-4257-4da4-9de6-458b5ec1994d", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:54:54.387057 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-9a33bfec-30d9-4d7d-8c2b-1b498dcecadf {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:54:54.387767 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-30a27fad-6c51-4ac7-b344-2cbdb5c05162 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:54.388017 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '982ed1f8-4257-4da4-9de6-458b5ec1994d', 'code': 200} May 31 11:54:54.472258 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:54.476178 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Pushing event updated for resources: {'Port': ['ID=3670cfae-0d92-4d02-a5ba-5c373efc785e,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:54.482212 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:54.506830 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Network c36bb4a9-2822-49b1-b291-69604a881999 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:54.509121 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:54.518948 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-8fc71910-652b-4185-a659-882d54470e02 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Unset dirty status for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:network {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:54.519123 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:54.519931 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-8fc71910-652b-4185-a659-882d54470e02 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 816 time: 0.8589604 May 31 11:54:54.535185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:54.535851 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:54.605329 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Transition to ACTIVE for port object d447eadc-67c7-4eca-8295-e5ae670ce616 will not be triggered until provisioned by entity DHCP. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:54.608919 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:54.644574 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.4185898 May 31 11:54:54.652291 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ff98c9d7-9f46-48f5-b6a9-f4ae3914cc62 req-2c7e99aa-8409-4e0c-a57e-bc608666aff1 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 210 time: 0.0520396 May 31 11:54:54.652748 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f21012f6-9cfa-442f-b326-42af02da004c req-35064419-c942-4662-87d5-d2c1506cb79e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:54.654212 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-35064419-c942-4662-87d5-d2c1506cb79e service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.1775424 May 31 11:54:54.657373 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Request body: {'port': {'device_id': '71751849-4dff-4706-b4fe-d196e7c2e6e2', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:54.662105 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-936b5759-266b-4750-ab91-1fe25a5bce3f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Synchronizing usage tracker for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:security_group {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:54.724065 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-5ba37a77-b53d-40aa-9728-fc969aaa62b5 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0580771 May 31 11:54:54.777142 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69e57490-14b9-4db4-ada4-e23136dcaabf req-bbfe378b-a7e1-4f13-94b1-883f096e1fcc tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:54.786216 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-69e57490-14b9-4db4-ada4-e23136dcaabf req-bbfe378b-a7e1-4f13-94b1-883f096e1fcc tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 943 time: 0.4046068 May 31 11:54:54.817752 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.47', 49585) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:54.856428 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:54.862523 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-936b5759-266b-4750-ab91-1fe25a5bce3f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Unset dirty status for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:security_group {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:54.863609 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-936b5759-266b-4750-ab91-1fe25a5bce3f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d HTTP/1.1" status: 200 len: 379 time: 0.3329768 May 31 11:54:54.883895 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'port': {'device_id': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'admin_state_up': True, 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'security_groups': ['8e2d5996-59e7-4c4a-9bfd-8eee9c767259']}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:54.917189 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:54.939661 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-69e57490-14b9-4db4-ada4-e23136dcaabf req-17b27c0b-8b1a-43e4-a816-c4a0ab0469fd tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1065202 May 31 11:54:55.001969 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-420b21f1-614c-4d10-93c4-9dae558ff91a service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:55.131324 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-500c8bbc-e91f-4a20-b479-dac018e6e951 req-46f923f8-be11-48a3-af48-d5a512769ff6 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:55.132920 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-500c8bbc-e91f-4a20-b479-dac018e6e951 req-46f923f8-be11-48a3-af48-d5a512769ff6 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 945 time: 0.3563848 May 31 11:54:55.147268 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-238e0181-d412-4254-803f-ffcfd0fc4fad service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4140320 May 31 11:54:55.166822 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:55.167026 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-2d1b6e9a-0705-47f5-a6d2-658beda7fbe1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:55.168281 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-2d1b6e9a-0705-47f5-a6d2-658beda7fbe1 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 1098 time: 0.3481278 May 31 11:54:55.171094 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:55.210352 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:55.216692 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:55.217338 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.221647 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'Port': ['ID=d447eadc-67c7-4eca-8295-e5ae670ce616,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:55.236189 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.233s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:55.263721 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.371660 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-500c8bbc-e91f-4a20-b479-dac018e6e951 req-8b5eb4c0-caa1-4f3c-9c6b-72986da7e960 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2315850 May 31 11:54:55.374748 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:55.466638 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:55.522465 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:55.544949 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device c4a233f2-9446-42a7-964d-1dc2496c938a details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:54:55.549356 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e977728-5020-4985-9a9b-524cf225ba61 req-d89c13b0-c308-430e-83ff-432bd604941d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:55.551102 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9e977728-5020-4985-9a9b-524cf225ba61 req-d89c13b0-c308-430e-83ff-432bd604941d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 931 time: 0.4574964 May 31 11:54:55.560501 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.9229233 May 31 11:54:55.576654 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-d9e66413-7ca4-487d-831d-3e33c7d11fd4 None None] Provisioning for port d447eadc-67c7-4eca-8295-e5ae670ce616 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:55.603300 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Usage tracker for resource:security_group_rule and tenant:8832087fe3ac414e826d6b2f552ef7cd is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:55.603406 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-d9e66413-7ca4-487d-831d-3e33c7d11fd4 None None] Provisioning complete for port d447eadc-67c7-4eca-8295-e5ae670ce616 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:55.603406 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-d9e66413-7ca4-487d-831d-3e33c7d11fd4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.603573 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Request body: {'port': {'device_id': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:55.649777 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:55.656012 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-55233c9f-7f8d-4f65-bc31-4b2480558430 req-3471209b-7594-4cb3-b684-9b2a7c5b4f8f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 210 time: 0.0686700 May 31 11:54:55.660967 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:55.666569 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 3670cfae-0d92-4d02-a5ba-5c373efc785e {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:54:55.675042 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-b3ab07c8-afb3-4f12-87c9-d4470c04f48f service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5115080 May 31 11:54:55.678436 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.700218 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-d9e66413-7ca4-487d-831d-3e33c7d11fd4 None None] Port d447eadc-67c7-4eca-8295-e5ae670ce616 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:54:55.712244 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Transition to ACTIVE for port object 3670cfae-0d92-4d02-a5ba-5c373efc785e will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:55.779824 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9e977728-5020-4985-9a9b-524cf225ba61 req-2096ed03-bb43-4059-82d7-cf7b15c537b3 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2158117 May 31 11:54:55.786570 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.829088 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.844946 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Made reservation on behalf of 0e7f0a333ee54b31bcd5c2290c3e284d for: {'port': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:54:55.848339 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.873075 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.881714 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port c4a233f2-9446-42a7-964d-1dc2496c938a, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c4a233f2-9446-42a7-964d-1dc2496c938a,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:55.888559 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1f508b2c-c666-43f8-a274-65efd24ff598 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:55.900388 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'c4a233f2-9446-42a7-964d-1dc2496c938a', 'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'port_id': 'c4a233f2-9446-42a7-964d-1dc2496c938a', 'mac_address': 'fa:16:3e:04:f4:25', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 356, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c58bc8a-d411-461e-a483-39773ff0d6f4', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:54:55.969024 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:55.990346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id c4a233f2-9446-42a7-964d-1dc2496c938a {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:54:56.023721 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.028684 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port c4a233f2-9446-42a7-964d-1dc2496c938a, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c4a233f2-9446-42a7-964d-1dc2496c938a,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:56.057777 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Pushing event updated for resources: {'Port': ['ID=3670cfae-0d92-4d02-a5ba-5c373efc785e,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:56.064981 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.177s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:56.067780 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-79e58b1b-bd05-4da2-a0f1-1c16643b77f3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.014s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:56.104800 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c1c3dc53-c77e-41e0-8f3b-a62960b91022 req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.9273152 May 31 11:54:56.133201 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.134504 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.136204 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:56.154808 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Attempting to bind port 3670cfae-0d92-4d02-a5ba-5c373efc785e on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:56.155146 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Attempting to bind port 3670cfae-0d92-4d02-a5ba-5c373efc785e by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:56.155475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Attempting to bind port 3670cfae-0d92-4d02-a5ba-5c373efc785e on network c36bb4a9-2822-49b1-b291-69604a881999 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:56.160652 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Pushing event updated for resources: {'SecurityGroup': ['ID=2eb670cf-ad32-4fa0-b6dc-056dbd3bff8e,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:56.163937 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:56.173693 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b482c668-9e3f-4803-8271-1242dcc6e56b req-cdeef2c4-1ca9-46fb-aee5-70115f3a5d98 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:56.178215 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b482c668-9e3f-4803-8271-1242dcc6e56b req-cdeef2c4-1ca9-46fb-aee5-70115f3a5d98 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 931 time: 0.3381197 May 31 11:54:56.180776 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Port 3670cfae-0d92-4d02-a5ba-5c373efc785e on network c36bb4a9-2822-49b1-b291-69604a881999 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:54:56.181134 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Attempting to bind port 3670cfae-0d92-4d02-a5ba-5c373efc785e on network c36bb4a9-2822-49b1-b291-69604a881999 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:56.222170 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:56.222943 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Checking segment: {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:56.223581 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Bound using segment: {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:56.224210 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Bound port: 3670cfae-0d92-4d02-a5ba-5c373efc785e, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '8a1730f3-5150-4597-9255-89922dd429cc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 387, 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:56.224746 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.234381 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=c4a233f2-9446-42a7-964d-1dc2496c938a,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:56.237459 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:56.290334 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Allocated IP 172.20.0.7 (19c09cdd-9420-4946-a105-024d5ea86ec3/4c55d678-c52f-4b1e-b005-86edad0fbf12/e0a8cb19-42fc-4a14-b86a-efda3d732b98) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:54:56.311509 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-b482c668-9e3f-4803-8271-1242dcc6e56b req-34665f1c-09ca-4d97-9460-48facf421ee0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:56.341203 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-b482c668-9e3f-4803-8271-1242dcc6e56b req-34665f1c-09ca-4d97-9460-48facf421ee0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:56.343047 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b482c668-9e3f-4803-8271-1242dcc6e56b req-34665f1c-09ca-4d97-9460-48facf421ee0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1578557 May 31 11:54:56.357426 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] For port 3670cfae-0d92-4d02-a5ba-5c373efc785e, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:56.361190 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] For port 3670cfae-0d92-4d02-a5ba-5c373efc785e, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:56.386200 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1104993a-b885-48c4-8274-2e991d840871 req-2a004c71-8048-4011-ab38-77b9a41bd329 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:56.387684 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1104993a-b885-48c4-8274-2e991d840871 req-2a004c71-8048-4011-ab38-77b9a41bd329 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 944 time: 0.2860096 May 31 11:54:56.428324 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['c4a233f2-9446-42a7-964d-1dc2496c938a'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:54:56.442434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.446814 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cce1d139-d29e-4383-bd39-0f71046a4a74 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:56.518228 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.db.resource_extend [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] It took 0.15 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 31 11:54:56.596493 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:56.597110 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.598608 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] 10.222.0.47 "PUT /v2.0/ports/3670cfae-0d92-4d02-a5ba-5c373efc785e HTTP/1.1" status: 200 len: 1093 time: 1.9452119 May 31 11:54:56.603820 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '71751849-4dff-4706-b4fe-d196e7c2e6e2', 'tag': '3670cfae-0d92-4d02-a5ba-5c373efc785e'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:54:56.606414 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-ee949f14-d1c8-4b05-a476-55316c7a556b" -d '{"events": [{"name": "network-changed", "server_uuid": "71751849-4dff-4706-b4fe-d196e7c2e6e2", "tag": "3670cfae-0d92-4d02-a5ba-5c373efc785e"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:54:56.631191 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Pushing event updated for resources: {'Port': ['ID=3670cfae-0d92-4d02-a5ba-5c373efc785e,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:56.644115 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:56.704152 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a298b89-588b-4296-a832-813e6dc90075 req-f6fe0c41-6f5d-461f-8941-92583814cf6d tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:56.717474 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6a298b89-588b-4296-a832-813e6dc90075 req-f6fe0c41-6f5d-461f-8941-92583814cf6d tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 945 time: 0.2203662 May 31 11:54:56.784765 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1104993a-b885-48c4-8274-2e991d840871 req-4b4a3887-9b53-4132-8bcd-dcce7076290e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3762593 May 31 11:54:56.794246 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id d447eadc-67c7-4eca-8295-e5ae670ce616 {{(pid=99039) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:54:56.814933 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.857004 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.858184 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Transition to ACTIVE for port object d447eadc-67c7-4eca-8295-e5ae670ce616 will not be triggered until provisioned by entity L2. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:56.873414 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Transition to ACTIVE for port object e0a8cb19-42fc-4a14-b86a-efda3d732b98 will not be triggered until provisioned by entity DHCP. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:56.902396 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6a298b89-588b-4296-a832-813e6dc90075 req-de4af88c-9838-452d-9776-cc8d92850dc8 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1816790 May 31 11:54:56.921028 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-99a3d991-95f5-44e1-84b8-d5f26fc16f7e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:56.922802 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-99a3d991-95f5-44e1-84b8-d5f26fc16f7e service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 1096 time: 0.1176233 May 31 11:54:56.933225 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:56.937409 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:56.988449 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:57.071758 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:54:56 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-f6799975-e7ee-468b-8d15-1f8788a0ef41 x-openstack-request-id: req-f6799975-e7ee-468b-8d15-1f8788a0ef41 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:54:57.079103 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "71751849-4dff-4706-b4fe-d196e7c2e6e2", "tag": "3670cfae-0d92-4d02-a5ba-5c373efc785e", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:54:57.079103 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-f6799975-e7ee-468b-8d15-1f8788a0ef41 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:54:57.079103 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '71751849-4dff-4706-b4fe-d196e7c2e6e2', 'tag': '3670cfae-0d92-4d02-a5ba-5c373efc785e', 'status': 'completed', 'code': 200} May 31 11:54:57.143418 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:54:57.150402 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:54:57.184285 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Pushing event updated for resources: {'Port': ['ID=d447eadc-67c7-4eca-8295-e5ae670ce616,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:57.186751 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:54:57.198628 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1012655f-d681-44d4-97e7-c74e878e24cd service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e HTTP/1.1" status: 200 len: 217 time: 0.2666037 May 31 11:54:57.199207 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:57.199873 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:57.268159 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.330s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:57.275378 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.181s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:57.314447 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-26f5da4f-d622-410d-92f7-62e5852f3da9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:57.316564 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-26f5da4f-d622-410d-92f7-62e5852f3da9 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.1084454 May 31 11:54:57.399400 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-e2502311-de53-44e4-adb7-ab91d5183419 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0745454 May 31 11:54:57.442028 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58180) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:57.453842 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Attempting to bind port d447eadc-67c7-4eca-8295-e5ae670ce616 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:57.454175 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Attempting to bind port d447eadc-67c7-4eca-8295-e5ae670ce616 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:57.454503 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Attempting to bind port d447eadc-67c7-4eca-8295-e5ae670ce616 on network 36cce22a-502c-464d-a9ac-50f2bc247e07 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:57.476618 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.6057549 May 31 11:54:57.488552 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b853f52-6d8d-4492-882e-25b2a3d06b6e req-925eb3b6-40d4-4e7f-8139-0405275c0303 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:57.496800 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8b853f52-6d8d-4492-882e-25b2a3d06b6e req-925eb3b6-40d4-4e7f-8139-0405275c0303 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 943 time: 0.3481424 May 31 11:54:57.498830 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Port d447eadc-67c7-4eca-8295-e5ae670ce616 on network 36cce22a-502c-464d-a9ac-50f2bc247e07 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:54:57.499157 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Attempting to bind port d447eadc-67c7-4eca-8295-e5ae670ce616 on network 36cce22a-502c-464d-a9ac-50f2bc247e07 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:57.518005 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:57.521422 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Checking segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:57.521807 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Bound using segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:57.522277 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Bound port: d447eadc-67c7-4eca-8295-e5ae670ce616, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'}}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:57.522630 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:57.541611 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device c4a233f2-9446-42a7-964d-1dc2496c938a up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:54:57.546760 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Request body: {'port': {'device_id': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:54:57.664711 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce8012ce-f4ef-4f9f-9793-825377bed19a req-0674a454-d500-4b98-96c2-40b9619621e0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:57.666608 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce8012ce-f4ef-4f9f-9793-825377bed19a req-0674a454-d500-4b98-96c2-40b9619621e0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 931 time: 0.5536535 May 31 11:54:57.674438 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:57.692546 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.418s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:57.704444 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-98c7e502-762d-4349-87a7-ae03a5842988 None None] Provisioning for port e0a8cb19-42fc-4a14-b86a-efda3d732b98 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:57.708632 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port c4a233f2-9446-42a7-964d-1dc2496c938a completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:54:57.712781 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-98c7e502-762d-4349-87a7-ae03a5842988 None None] Provisioning complete for port e0a8cb19-42fc-4a14-b86a-efda3d732b98 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:57.715165 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8b853f52-6d8d-4492-882e-25b2a3d06b6e req-26d5529e-3e0c-4c11-85a1-ed716a0e43ac tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2152004 May 31 11:54:57.715285 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-98c7e502-762d-4349-87a7-ae03a5842988 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:57.717475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:57.747174 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port c4a233f2-9446-42a7-964d-1dc2496c938a triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:54:57.748773 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:57.762832 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:54:57.783256 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce8012ce-f4ef-4f9f-9793-825377bed19a req-089579a9-dd0f-4dfd-a833-192b4a3037d1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0952370 May 31 11:54:57.784265 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Usage tracker for resource:security_group_rule and tenant:e7a836f17b81435389b8ebce3b4b090d is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:54:57.788646 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:54:57.793851 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-1d396e63-a8bc-49ff-a1de-00451ab1b7bd service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3825154 May 31 11:54:57.798968 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:54:57.806050 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:57.887019 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] For port d447eadc-67c7-4eca-8295-e5ae670ce616, host hv-cinder-79123, cleared binding levels {{(pid=99039) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:57.893363 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] For port d447eadc-67c7-4eca-8295-e5ae670ce616, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=d447eadc-67c7-4eca-8295-e5ae670ce616,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99039) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:57.992575 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-98c7e502-762d-4349-87a7-ae03a5842988 None None] Port e0a8cb19-42fc-4a14-b86a-efda3d732b98 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:54:58.065315 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.070696 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.086767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:58.109277 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bb55be8b-198d-40e7-849a-8543fbf35000', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c4a233f2-9446-42a7-964d-1dc2496c938a'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:54:58.111830 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-a3a6eae1-4f4f-4a3b-a5b8-cc6603c0a301" -d '{"events": [{"server_uuid": "bb55be8b-198d-40e7-849a-8543fbf35000", "name": "network-vif-plugged", "status": "completed", "tag": "c4a233f2-9446-42a7-964d-1dc2496c938a"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:54:58.120142 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port c4a233f2-9446-42a7-964d-1dc2496c938a, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c4a233f2-9446-42a7-964d-1dc2496c938a,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:54:58.190440 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.192872 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-ac53e6ab-8128-44c3-8987-dbdef84bc871 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:58.238893 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9476fe7e-3530-4734-8e39-5fd8f7ba4c84 req-c7f306ed-140d-4818-9331-657177dcbd50 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4353952 May 31 11:54:58.245385 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:58.246163 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.247794 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] 10.222.0.47 "PUT /v2.0/ports/d447eadc-67c7-4eca-8295-e5ae670ce616 HTTP/1.1" status: 200 len: 1093 time: 2.6658294 May 31 11:54:58.248241 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'tag': 'd447eadc-67c7-4eca-8295-e5ae670ce616'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:54:58.250707 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-d0511567-f08d-4465-a7d6-a590e68f270e" -d '{"events": [{"name": "network-changed", "server_uuid": "f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6", "tag": "d447eadc-67c7-4eca-8295-e5ae670ce616"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:54:58.272884 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.277208 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Pushing event updated for resources: {'Port': ['ID=d447eadc-67c7-4eca-8295-e5ae670ce616,revision_number=3']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:58.282576 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-13d8b974-1f0f-421d-ab8c-35064907cea6 req-c7a6cb18-d047-4d0d-bc39-81f60b7773d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:58.284086 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-13d8b974-1f0f-421d-ab8c-35064907cea6 req-c7a6cb18-d047-4d0d-bc39-81f60b7773d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 944 time: 0.2190058 May 31 11:54:58.311706 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=c4a233f2-9446-42a7-964d-1dc2496c938a,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:58.315556 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.229s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:58.317346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.124s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:58.350408 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-13d8b974-1f0f-421d-ab8c-35064907cea6 req-70cf9c36-e73a-4406-b0aa-ee033799fd7b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0570545 May 31 11:54:58.351782 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.391334 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:58.418709 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76a07d1b-2f9f-4a3a-95fc-c2581d408c0e req-5ea643e9-1435-4337-92b6-eac45172d4ef tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:58.419518 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-76a07d1b-2f9f-4a3a-95fc-c2581d408c0e req-5ea643e9-1435-4337-92b6-eac45172d4ef tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:58.424429 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=e7a836f17b81435389b8ebce3b4b090d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.9808567 May 31 11:54:58.455608 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3ce2cd97-2da8-4ace-99d3-2c7d618b6b20,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:58.458155 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58200) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:54:58.472070 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:58.572626 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-fd2f1ee1-f79c-4608-9e1e-bd63a0986bdf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:58.574175 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-fd2f1ee1-f79c-4608-9e1e-bd63a0986bdf service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 1096 time: 0.1439672 May 31 11:54:58.577000 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-76a07d1b-2f9f-4a3a-95fc-c2581d408c0e req-5ea643e9-1435-4337-92b6-eac45172d4ef tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:58.578021 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-76a07d1b-2f9f-4a3a-95fc-c2581d408c0e req-5ea643e9-1435-4337-92b6-eac45172d4ef tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 943 time: 0.6155844 May 31 11:54:58.620794 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-de6c7a20-baac-40f2-9e3e-cd8d52d0b6d7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.653478 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-9ed542c1-fec4-4483-9a7d-47a894148236 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:58.655313 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-9ed542c1-fec4-4483-9a7d-47a894148236 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 1096 time: 0.2646406 May 31 11:54:58.660595 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-de6c7a20-baac-40f2-9e3e-cd8d52d0b6d7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.701815 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-e747e4ba-1216-4c24-b592-6b0d94351009 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=d447eadc-67c7-4eca-8295-e5ae670ce616 HTTP/1.1" status: 200 len: 217 time: 0.1114712 May 31 11:54:58.764224 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-13f86f14-e7b2-4fe5-a8ec-f213251a8254 req-511b16d5-a69e-4d77-b1b7-e714eefeb721 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:58.766076 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-13f86f14-e7b2-4fe5-a8ec-f213251a8254 req-511b16d5-a69e-4d77-b1b7-e714eefeb721 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 945 time: 0.2816966 May 31 11:54:58.785243 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e0a8cb19-42fc-4a14-b86a-efda3d732b98 {{(pid=99039) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:54:58.792536 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-76a07d1b-2f9f-4a3a-95fc-c2581d408c0e req-3d06ea9f-2480-434a-945f-283d6cdb693c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2035632 May 31 11:54:58.799839 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.811226 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-ba54a3d1-6c5e-476b-b910-2b598aeb7314 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:58.812971 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-ba54a3d1-6c5e-476b-b910-2b598aeb7314 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0979784 May 31 11:54:58.860410 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-13f86f14-e7b2-4fe5-a8ec-f213251a8254 req-7984164c-a96b-42dd-a6d2-dfd4a612c5ed tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0860126 May 31 11:54:58.875304 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:54:58 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-37330769-fd67-41d8-996b-a655cfa70f84 x-openstack-request-id: req-37330769-fd67-41d8-996b-a655cfa70f84 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:54:58.875832 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "bb55be8b-198d-40e7-849a-8543fbf35000", "name": "network-vif-plugged", "status": "completed", "tag": "c4a233f2-9446-42a7-964d-1dc2496c938a", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:54:58.875925 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-37330769-fd67-41d8-996b-a655cfa70f84 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:54:58.876262 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bb55be8b-198d-40e7-849a-8543fbf35000', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c4a233f2-9446-42a7-964d-1dc2496c938a', 'code': 200} May 31 11:54:58.878931 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-52ca9e1a-9a6d-4759-80d3-bc8db0196eab service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:58.880421 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-52ca9e1a-9a6d-4759-80d3-bc8db0196eab service neutron] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 920 time: 0.2125800 May 31 11:54:58.984505 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-de6c7a20-baac-40f2-9e3e-cd8d52d0b6d7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:58.996399 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:54:58 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-77a711f1-e8fc-4858-9318-39886c03d90e x-openstack-request-id: req-77a711f1-e8fc-4858-9318-39886c03d90e {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:54:58.996706 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6", "tag": "d447eadc-67c7-4eca-8295-e5ae670ce616", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:54:58.996971 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-77a711f1-e8fc-4858-9318-39886c03d90e {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:54:58.997111 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-c73eab01-d2c1-4827-a9c0-3f605dd8f08c service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e HTTP/1.1" status: 200 len: 217 time: 0.1056519 May 31 11:54:58.997369 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'tag': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'status': 'completed', 'code': 200} May 31 11:54:58.998731 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:59.035174 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Transition to ACTIVE for port object e0a8cb19-42fc-4a14-b86a-efda3d732b98 will not be triggered until provisioned by entity L2. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:54:59.049420 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de6c7a20-baac-40f2-9e3e-cd8d52d0b6d7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3ce2cd97-2da8-4ace-99d3-2c7d618b6b20,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:59.054445 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-de6c7a20-baac-40f2-9e3e-cd8d52d0b6d7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:59.060138 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-95272bff-2f71-4561-85d1-dca9a80b0fb8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:59.061847 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-95272bff-2f71-4561-85d1-dca9a80b0fb8 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2419717 May 31 11:54:59.069091 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-de6c7a20-baac-40f2-9e3e-cd8d52d0b6d7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:59.073055 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-de6c7a20-baac-40f2-9e3e-cd8d52d0b6d7 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/3ce2cd97-2da8-4ace-99d3-2c7d618b6b20 HTTP/1.1" status: 204 len: 168 time: 0.6063993 May 31 11:54:59.116293 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-a98d2f65-d1b9-40c1-96e5-cc2f48d73c69 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:59.117191 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-a98d2f65-d1b9-40c1-96e5-cc2f48d73c69 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.1117852 May 31 11:54:59.146019 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:59.252030 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-01488356-57ef-49dc-b4e9-031f6a4b4609 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0662346 May 31 11:54:59.377410 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:54:59.474809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-733a23cd-f4bc-4637-80ee-e996976dec25 req-0692989e-d3c1-45d8-bb70-0cc855d35332 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:59.476790 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-733a23cd-f4bc-4637-80ee-e996976dec25 req-0692989e-d3c1-45d8-bb70-0cc855d35332 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Synchronizing usage tracker for tenant:c94978b38250481095d2824adfb16ed4 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:59.551271 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-a1345a65-cc98-405e-88c6-020fd97b884f service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2885787 May 31 11:54:59.558845 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device f18f632c-d53b-4423-93a7-f93c3e8e0778 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:54:59.627925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f450cd4-462e-4b8a-b840-1531a9970752 req-c6eabd05-b4fb-499e-befb-fd4d0afc34cf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:59.629272 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2f450cd4-462e-4b8a-b840-1531a9970752 req-c6eabd05-b4fb-499e-befb-fd4d0afc34cf service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 1098 time: 0.2215474 May 31 11:54:59.630817 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:54:59.645346 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.257s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:54:59.684538 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-733a23cd-f4bc-4637-80ee-e996976dec25 req-0692989e-d3c1-45d8-bb70-0cc855d35332 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Unset dirty status for tenant:c94978b38250481095d2824adfb16ed4 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:54:59.685751 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-733a23cd-f4bc-4637-80ee-e996976dec25 req-0692989e-d3c1-45d8-bb70-0cc855d35332 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 943 time: 0.5725002 May 31 11:54:59.689549 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-66ccdc16-fca7-424b-9a43-b1f416626f0c service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.6092830 May 31 11:54:59.691167 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Attempting to bind port e0a8cb19-42fc-4a14-b86a-efda3d732b98 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:54:59.691496 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Attempting to bind port e0a8cb19-42fc-4a14-b86a-efda3d732b98 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:54:59.691816 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Attempting to bind port e0a8cb19-42fc-4a14-b86a-efda3d732b98 on network 19c09cdd-9420-4946-a105-024d5ea86ec3 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:59.724827 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Port e0a8cb19-42fc-4a14-b86a-efda3d732b98 on network 19c09cdd-9420-4946-a105-024d5ea86ec3 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:54:59.725285 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Attempting to bind port e0a8cb19-42fc-4a14-b86a-efda3d732b98 on network 19c09cdd-9420-4946-a105-024d5ea86ec3 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:54:59.742698 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 54, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:54:59.744100 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Checking segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:54:59.744100 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Bound using segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:54:59.745027 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Bound port: e0a8cb19-42fc-4a14-b86a-efda3d732b98, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'}}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:54:59.745400 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:54:59.827657 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c1b825cb-a82e-43e4-a2aa-17fd159377fe req-af289083-434a-4661-9f87-b2e40ee8b17d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:54:59.828358 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-c1b825cb-a82e-43e4-a2aa-17fd159377fe req-af289083-434a-4661-9f87-b2e40ee8b17d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Synchronizing usage tracker for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:59.835295 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-2f450cd4-462e-4b8a-b840-1531a9970752 req-b2746ec0-087e-4c5d-883c-1f0140f91965 service neutron] Synchronizing usage tracker for tenant:8832087fe3ac414e826d6b2f552ef7cd on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:59.858150 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-733a23cd-f4bc-4637-80ee-e996976dec25 req-50b11cb9-659e-4ed7-8db2-4279a5c0e59f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Synchronizing usage tracker for tenant:c94978b38250481095d2824adfb16ed4 on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:54:59.945838 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, cleared binding levels {{(pid=99039) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:54:59.948866 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99039) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:54:59.990346 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-e707c5ea-800d-4563-ae3d-0ce57ed9d30a service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4149859 May 31 11:54:59.990953 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75d55a0c-694b-4e6c-81e2-f35cc536885b req-9fbec197-4f2c-4c8d-a508-ff7f8c6961d1 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2940531 May 31 11:54:59.991427 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port f18f632c-d53b-4423-93a7-f93c3e8e0778, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:00.005261 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'f18f632c-d53b-4423-93a7-f93c3e8e0778', 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999', 'port_id': 'f18f632c-d53b-4423-93a7-f93c3e8e0778', 'mac_address': 'fa:16:3e:e4:7f:6a', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 387, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '6312c1ab-b503-4bcc-af02-8274dbf2679c', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:55:00.012914 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-2f450cd4-462e-4b8a-b840-1531a9970752 req-b2746ec0-087e-4c5d-883c-1f0140f91965 service neutron] Unset dirty status for tenant:8832087fe3ac414e826d6b2f552ef7cd on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:55:00.014043 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2f450cd4-462e-4b8a-b840-1531a9970752 req-b2746ec0-087e-4c5d-883c-1f0140f91965 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3274124 May 31 11:55:00.064948 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-c1b825cb-a82e-43e4-a2aa-17fd159377fe req-af289083-434a-4661-9f87-b2e40ee8b17d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Unset dirty status for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:55:00.066560 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c1b825cb-a82e-43e4-a2aa-17fd159377fe req-af289083-434a-4661-9f87-b2e40ee8b17d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.9216275 May 31 11:55:00.067970 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:00.072116 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:00.073577 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-733a23cd-f4bc-4637-80ee-e996976dec25 req-50b11cb9-659e-4ed7-8db2-4279a5c0e59f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Unset dirty status for tenant:c94978b38250481095d2824adfb16ed4 on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:55:00.089672 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:00.097175 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-733a23cd-f4bc-4637-80ee-e996976dec25 req-50b11cb9-659e-4ed7-8db2-4279a5c0e59f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3813736 May 31 11:55:00.097880 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58228) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:00.126142 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id f18f632c-d53b-4423-93a7-f93c3e8e0778 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:55:00.174087 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port f18f632c-d53b-4423-93a7-f93c3e8e0778, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:00.209375 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:00.213467 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:00.214285 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:00.217237 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] 10.222.0.47 "PUT /v2.0/ports/e0a8cb19-42fc-4a14-b86a-efda3d732b98 HTTP/1.1" status: 200 len: 1093 time: 2.7027907 May 31 11:55:00.218307 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=3']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:00.236693 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c1b825cb-a82e-43e4-a2aa-17fd159377fe req-6eed5c94-4d4b-470e-9215-602adce07bb0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1623423 May 31 11:55:00.239750 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fa0c3fc6-9257-4e42-8bd9-9bd0f17561b5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.150s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:00.269911 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:00.272315 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:00.393016 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:00.413811 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Usage tracker for resource:security_group_rule and tenant:cd7eefad97794b868c18220a00b23282 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:00.423344 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:00.440013 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:00.455431 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=f18f632c-d53b-4423-93a7-f93c3e8e0778,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:00.483888 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.211s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:00.497270 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0511567-f08d-4465-a7d6-a590e68f270e req-12a5c51d-437a-40b9-9a56-1be39caaa52f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:00.498832 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0511567-f08d-4465-a7d6-a590e68f270e req-12a5c51d-437a-40b9-9a56-1be39caaa52f service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 1096 time: 0.3391531 May 31 11:55:00.548343 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['f18f632c-d53b-4423-93a7-f93c3e8e0778'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:55:00.551837 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c0b535f-4a5e-44ef-a1e6-decd1bf81b7b req-3261f7f2-418a-442c-b681-370a8f6dc2ba tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:00.553298 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2c0b535f-4a5e-44ef-a1e6-decd1bf81b7b req-3261f7f2-418a-442c-b681-370a8f6dc2ba tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 943 time: 0.1144400 May 31 11:55:00.553759 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-135b0ce4-e459-4d9e-a7c9-a064e0dca8e5 req-c307d7a6-e8dc-4387-85d1-1a41112e2de6 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:00.555292 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-135b0ce4-e459-4d9e-a7c9-a064e0dca8e5 req-c307d7a6-e8dc-4387-85d1-1a41112e2de6 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 945 time: 0.2947299 May 31 11:55:00.557975 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-f1bdd699-44ca-4b59-98c7-ef844b28393b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:00.559350 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-f1bdd699-44ca-4b59-98c7-ef844b28393b service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 1096 time: 0.2278976 May 31 11:55:00.675265 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2c0b535f-4a5e-44ef-a1e6-decd1bf81b7b req-0cb4bd2b-e208-4442-91f1-3dad77836788 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1143708 May 31 11:55:00.677019 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-417fac7e-1634-4cff-9718-ba814b66830c service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98 HTTP/1.1" status: 200 len: 217 time: 0.1000407 May 31 11:55:00.761299 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-5e0a18cb-2ce9-4dd9-9e92-50d29e64a21a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:00.762817 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-5e0a18cb-2ce9-4dd9-9e92-50d29e64a21a service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0789130 May 31 11:55:00.822669 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-135b0ce4-e459-4d9e-a7c9-a064e0dca8e5 req-6059af5e-1b5e-4ea0-83c7-c1c5f4d28825 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1145611 May 31 11:55:00.879346 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:00.880927 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0511567-f08d-4465-a7d6-a590e68f270e req-77aaff6d-08d1-48b9-8dcd-9c22c6fc399e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:00.882305 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0511567-f08d-4465-a7d6-a590e68f270e req-77aaff6d-08d1-48b9-8dcd-9c22c6fc399e service neutron] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 912 time: 0.3747132 May 31 11:55:00.882422 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-108ca526-5013-4295-a62c-57fbc7b43d3a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:00.883772 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-108ca526-5013-4295-a62c-57fbc7b43d3a service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1129527 May 31 11:55:00.977341 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0511567-f08d-4465-a7d6-a590e68f270e req-b3ca9ccd-b961-4b7c-a7c0-d06646a0d61f service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=d447eadc-67c7-4eca-8295-e5ae670ce616 HTTP/1.1" status: 200 len: 217 time: 0.0821495 May 31 11:55:01.006740 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:01.009002 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-74ea58af-79ea-4d31-bd22-733fc624112b" -d '{"events": [{"name": "network-changed", "server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:01.036723 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-bee7dffe-3ffa-4441-84d4-9048d6f10815 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1413319 May 31 11:55:01.055569 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0511567-f08d-4465-a7d6-a590e68f270e req-ec00fb2c-a124-453f-8ebc-ec540f584212 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:01.058097 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0511567-f08d-4465-a7d6-a590e68f270e req-ec00fb2c-a124-453f-8ebc-ec540f584212 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0745687 May 31 11:55:01.067103 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:55:01 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-7bcb64be-be0e-4441-8ed6-620251720e48 x-openstack-request-id: req-7bcb64be-be0e-4441-8ed6-620251720e48 {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:01.067484 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:01.067819 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-7bcb64be-be0e-4441-8ed6-620251720e48 {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:01.068281 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'status': 'completed', 'code': 200} May 31 11:55:01.070362 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:01.081744 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:01.164956 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d5852371-12a5-4102-bfe4-e8a67337ecec,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:01.170871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:01.178048 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=cd7eefad97794b868c18220a00b23282&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0692191 May 31 11:55:01.179986 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 58260) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:01.213018 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0511567-f08d-4465-a7d6-a590e68f270e req-c2acff2e-b9e0-4604-a453-a49cfb8b5291 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:01.214975 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0511567-f08d-4465-a7d6-a590e68f270e req-c2acff2e-b9e0-4604-a453-a49cfb8b5291 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1357534 May 31 11:55:01.220621 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a57f9137-3a34-487e-95ca-ab93ea6a462a req-baa53beb-c1e5-41c3-bbd2-99e8e788d806 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1756418 May 31 11:55:01.269456 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8e3bb02a-fa60-4553-98a0-142b8d51e464 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:01.322931 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8e3bb02a-fa60-4553-98a0-142b8d51e464 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:01.487104 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a416af99-db8b-4642-b804-9155a87fba2c req-c7854615-6685-487e-9c6c-7c1f34f5d0a2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:01.488759 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a416af99-db8b-4642-b804-9155a87fba2c req-c7854615-6685-487e-9c6c-7c1f34f5d0a2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 943 time: 0.2715690 May 31 11:55:01.496337 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0511567-f08d-4465-a7d6-a590e68f270e req-59a3fff1-c6a5-4169-913c-a835f33bc76a service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2398005 May 31 11:55:01.575831 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 3670cfae-0d92-4d02-a5ba-5c373efc785e details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:55:01.607001 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74ea58af-79ea-4d31-bd22-733fc624112b req-265a4da2-6898-46aa-9c5f-026d5585f92b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:01.608643 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74ea58af-79ea-4d31-bd22-733fc624112b req-265a4da2-6898-46aa-9c5f-026d5585f92b service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 1096 time: 0.1492825 May 31 11:55:01.630295 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device f18f632c-d53b-4423-93a7-f93c3e8e0778 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:55:01.658509 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a416af99-db8b-4642-b804-9155a87fba2c req-00d5a093-07a5-4f66-803e-fcb0f88e39a3 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1609640 May 31 11:55:01.708826 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd1b9766-07ab-40d7-9bef-3f4563c5d5ee req-54bd7947-e0c7-40d9-96d0-f7634abb6ca5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:01.710340 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-cd1b9766-07ab-40d7-9bef-3f4563c5d5ee req-54bd7947-e0c7-40d9-96d0-f7634abb6ca5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.3014138 May 31 11:55:01.728673 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0511567-f08d-4465-a7d6-a590e68f270e req-330f1f45-2e1f-4ffe-b0f6-99d42ce440f2 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2135870 May 31 11:55:01.782742 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8e3bb02a-fa60-4553-98a0-142b8d51e464 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:01.798710 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 3670cfae-0d92-4d02-a5ba-5c373efc785e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:01.815619 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f3a49e2a-c2c7-4897-8986-0515c2abae3c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:01.829881 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '3670cfae-0d92-4d02-a5ba-5c373efc785e', 'network_id': 'c36bb4a9-2822-49b1-b291-69604a881999', 'port_id': '3670cfae-0d92-4d02-a5ba-5c373efc785e', 'mac_address': 'fa:16:3e:2e:74:87', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 387, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '6312c1ab-b503-4bcc-af02-8274dbf2679c', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:55:01.851532 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8e3bb02a-fa60-4553-98a0-142b8d51e464 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d5852371-12a5-4102-bfe4-e8a67337ecec,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:01.856188 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8e3bb02a-fa60-4553-98a0-142b8d51e464 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:01.862366 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-cd1b9766-07ab-40d7-9bef-3f4563c5d5ee req-23e19a6e-9a70-4886-9485-99468c5c6ad9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1220472 May 31 11:55:01.870906 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8e3bb02a-fa60-4553-98a0-142b8d51e464 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:01.871761 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8e3bb02a-fa60-4553-98a0-142b8d51e464 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/d5852371-12a5-4102-bfe4-e8a67337ecec HTTP/1.1" status: 204 len: 168 time: 0.6892691 May 31 11:55:01.873245 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74ea58af-79ea-4d31-bd22-733fc624112b req-f6524a55-1aea-422b-9034-6f90432b0cb4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:01.874997 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74ea58af-79ea-4d31-bd22-733fc624112b req-f6524a55-1aea-422b-9034-6f90432b0cb4 service neutron] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 917 time: 0.2526464 May 31 11:55:01.937014 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port f18f632c-d53b-4423-93a7-f93c3e8e0778 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:55:01.976812 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port f18f632c-d53b-4423-93a7-f93c3e8e0778 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:55:01.977240 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:02.069505 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74ea58af-79ea-4d31-bd22-733fc624112b req-8b778c75-e0e0-4f90-98d2-7e62a389aea4 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98 HTTP/1.1" status: 200 len: 217 time: 0.1863265 May 31 11:55:02.091094 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:02.125651 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 3670cfae-0d92-4d02-a5ba-5c373efc785e {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:55:02.150913 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 3670cfae-0d92-4d02-a5ba-5c373efc785e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:02.175753 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74ea58af-79ea-4d31-bd22-733fc624112b req-1ea542be-152f-405e-8af9-e345758c035c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:02.177599 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74ea58af-79ea-4d31-bd22-733fc624112b req-1ea542be-152f-405e-8af9-e345758c035c service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0975118 May 31 11:55:02.214240 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:02.226805 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:02.231577 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:02.231703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19c6522b-68e7-4b20-b4fc-904a1412ac91 req-ccb8ea9d-e117-4209-8e04-c5315943d9c6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:02.233192 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-19c6522b-68e7-4b20-b4fc-904a1412ac91 req-ccb8ea9d-e117-4209-8e04-c5315943d9c6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 943 time: 0.2278183 May 31 11:55:02.316319 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port f18f632c-d53b-4423-93a7-f93c3e8e0778, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:02.320089 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '66c4d526-5295-45db-9e0b-516b82570603', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f18f632c-d53b-4423-93a7-f93c3e8e0778'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:02.322600 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-064169cd-7dba-4fba-b480-315a358f23b1" -d '{"events": [{"server_uuid": "66c4d526-5295-45db-9e0b-516b82570603", "name": "network-vif-plugged", "status": "completed", "tag": "f18f632c-d53b-4423-93a7-f93c3e8e0778"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:02.325314 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-19c6522b-68e7-4b20-b4fc-904a1412ac91 req-890a9643-b716-4ab6-90bd-7f0b0a0ece63 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0848622 May 31 11:55:02.363473 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=3670cfae-0d92-4d02-a5ba-5c373efc785e,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:02.365043 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74ea58af-79ea-4d31-bd22-733fc624112b req-bd5971cf-cd17-434d-9f62-7eacddeb913d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:02.366939 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74ea58af-79ea-4d31-bd22-733fc624112b req-bd5971cf-cd17-434d-9f62-7eacddeb913d service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1798205 May 31 11:55:02.369642 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.138s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:02.412508 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:02.414643 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:02.472350 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['3670cfae-0d92-4d02-a5ba-5c373efc785e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:55:02.561093 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=f18f632c-d53b-4423-93a7-f93c3e8e0778,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:02.574455 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.160s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:02.601030 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74ea58af-79ea-4d31-bd22-733fc624112b req-e0ef63e8-7c96-4cb1-900d-19074ce127e8 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2267113 May 31 11:55:02.719323 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-433e1b5b-bbf5-49ba-b45b-a0c8e060bd5b req-3a37b007-ddd4-4195-ba7a-e1fc5f050fb9 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:02.720772 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-433e1b5b-bbf5-49ba-b45b-a0c8e060bd5b req-3a37b007-ddd4-4195-ba7a-e1fc5f050fb9 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 945 time: 0.2379100 May 31 11:55:02.801084 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-433e1b5b-bbf5-49ba-b45b-a0c8e060bd5b req-6f99c994-b03c-4f47-86ac-7a4206c70a75 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0733724 May 31 11:55:02.804342 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74ea58af-79ea-4d31-bd22-733fc624112b req-928f4874-4634-4fcc-8418-34fbffd1e8c4 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1960838 May 31 11:55:02.859640 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:55:02 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-4ae0ce4b-5c6c-4a2e-81d9-c545f5635590 x-openstack-request-id: req-4ae0ce4b-5c6c-4a2e-81d9-c545f5635590 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:02.860228 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "66c4d526-5295-45db-9e0b-516b82570603", "name": "network-vif-plugged", "status": "completed", "tag": "f18f632c-d53b-4423-93a7-f93c3e8e0778", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:02.860228 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-4ae0ce4b-5c6c-4a2e-81d9-c545f5635590 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:02.860608 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '66c4d526-5295-45db-9e0b-516b82570603', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f18f632c-d53b-4423-93a7-f93c3e8e0778', 'code': 200} May 31 11:55:03.064785 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e19eaae-87f1-4c12-9016-2b7293bf5383 req-8d7da07c-9b94-4985-a5d5-e09225c291ba tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:03.066315 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1e19eaae-87f1-4c12-9016-2b7293bf5383 req-8d7da07c-9b94-4985-a5d5-e09225c291ba tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 944 time: 0.1223752 May 31 11:55:03.134816 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1e19eaae-87f1-4c12-9016-2b7293bf5383 req-b47b2942-eefc-446f-8c21-e01d56d57fe8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0622416 May 31 11:55:03.316973 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-65d04b7e-b286-40bf-86d1-e52fedf55543 req-cf4b184b-8f57-4ae0-8148-7badb1b3f1b0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:03.326272 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-65d04b7e-b286-40bf-86d1-e52fedf55543 req-cf4b184b-8f57-4ae0-8148-7badb1b3f1b0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1367517 May 31 11:55:03.387275 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66d08d99-70af-4574-967e-5da88fc7dfc3 req-6a39d06a-ad8e-4a65-ae8c-ae3fe08fdacd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:03.388053 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-65d04b7e-b286-40bf-86d1-e52fedf55543 req-0fd8abe7-c970-4650-bcb2-e38a4e52055a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0605977 May 31 11:55:03.399778 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66d08d99-70af-4574-967e-5da88fc7dfc3 req-6a39d06a-ad8e-4a65-ae8c-ae3fe08fdacd service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1099 time: 0.1270344 May 31 11:55:03.515003 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66d08d99-70af-4574-967e-5da88fc7dfc3 req-50d86bb0-6098-4a76-bfbb-e0bb396128b3 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1109734 May 31 11:55:03.570504 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device d447eadc-67c7-4eca-8295-e5ae670ce616 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:55:03.573118 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17fcc8e5-3751-4004-a0ff-15ba935bcea0 req-55372376-c2b9-481b-ac42-f0481d69c727 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:03.574921 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-17fcc8e5-3751-4004-a0ff-15ba935bcea0 req-55372376-c2b9-481b-ac42-f0481d69c727 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 943 time: 0.1061287 May 31 11:55:03.580540 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 3670cfae-0d92-4d02-a5ba-5c373efc785e up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:55:03.629610 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-17fcc8e5-3751-4004-a0ff-15ba935bcea0 req-2ecf5c7a-88f4-47ea-a81f-57fd39401d62 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0461471 May 31 11:55:03.771069 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port d447eadc-67c7-4eca-8295-e5ae670ce616, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=d447eadc-67c7-4eca-8295-e5ae670ce616,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:03.788622 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'port_id': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'mac_address': 'fa:16:3e:3a:c0:2b', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 363, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:55:03.857082 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:03.877466 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id d447eadc-67c7-4eca-8295-e5ae670ce616 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:55:03.885230 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 3670cfae-0d92-4d02-a5ba-5c373efc785e completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:55:03.896789 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 3670cfae-0d92-4d02-a5ba-5c373efc785e triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:55:03.897508 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:03.962801 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port d447eadc-67c7-4eca-8295-e5ae670ce616, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=d447eadc-67c7-4eca-8295-e5ae670ce616,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:04.029132 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:04.062512 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 3670cfae-0d92-4d02-a5ba-5c373efc785e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:04.076412 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:04.078269 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-c2670100-fd63-4325-a961-fd3956909425 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:04.144156 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:04.182728 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=d447eadc-67c7-4eca-8295-e5ae670ce616,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:04.199316 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:04.199828 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-2ac3db74-b629-4f69-a483-687e0b289069 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.047s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:04.278811 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=3670cfae-0d92-4d02-a5ba-5c373efc785e,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:04.281985 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:04.361808 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e806934b-dc02-45e0-b102-014419161de7 req-36832075-e2b6-4f28-979b-d9a339b6e222 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:04.369989 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e806934b-dc02-45e0-b102-014419161de7 req-36832075-e2b6-4f28-979b-d9a339b6e222 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1242645 May 31 11:55:04.402089 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['d447eadc-67c7-4eca-8295-e5ae670ce616'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:55:04.470144 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e806934b-dc02-45e0-b102-014419161de7 req-248dd26c-3842-4701-b7ed-13fe6bfd3c43 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0991135 May 31 11:55:04.676984 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbcc3722-a7a6-4a3b-80b4-72e81a2925cb req-2faee6bd-1c32-47ca-966a-ae73892ef688 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:04.679021 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fbcc3722-a7a6-4a3b-80b4-72e81a2925cb req-2faee6bd-1c32-47ca-966a-ae73892ef688 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0859840 May 31 11:55:04.734074 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fbcc3722-a7a6-4a3b-80b4-72e81a2925cb req-92d8e512-c96a-470e-ac64-e12beaabe981 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475702 May 31 11:55:04.862340 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '71751849-4dff-4706-b4fe-d196e7c2e6e2', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3670cfae-0d92-4d02-a5ba-5c373efc785e'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:04.863285 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a32a2f8-c64c-4f24-958d-c6703f7066be req-9df7b407-c985-4e0a-8645-5d3289f74fe4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:04.864809 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5a32a2f8-c64c-4f24-958d-c6703f7066be req-9df7b407-c985-4e0a-8645-5d3289f74fe4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 944 time: 0.0865970 May 31 11:55:04.864976 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-717a05d9-e0c7-4688-b19c-2fcfffb5fbb4" -d '{"events": [{"server_uuid": "71751849-4dff-4706-b4fe-d196e7c2e6e2", "name": "network-vif-plugged", "status": "completed", "tag": "3670cfae-0d92-4d02-a5ba-5c373efc785e"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:04.915055 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5a32a2f8-c64c-4f24-958d-c6703f7066be req-44ee31bc-e863-416b-9c86-70b8c148d75c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412228 May 31 11:55:04.917429 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:55:04 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-e5eed305-836f-4f12-aec2-2e3beabadd30 x-openstack-request-id: req-e5eed305-836f-4f12-aec2-2e3beabadd30 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:04.918299 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "71751849-4dff-4706-b4fe-d196e7c2e6e2", "name": "network-vif-plugged", "status": "completed", "tag": "3670cfae-0d92-4d02-a5ba-5c373efc785e", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:04.918942 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-e5eed305-836f-4f12-aec2-2e3beabadd30 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:04.919170 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '71751849-4dff-4706-b4fe-d196e7c2e6e2', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3670cfae-0d92-4d02-a5ba-5c373efc785e', 'code': 200} May 31 11:55:05.538557 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3c5d246-6a37-41d9-9e92-2caad2fa63cf req-7f9613b9-cf78-4890-80b2-26d67b913728 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:05.540020 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c3c5d246-6a37-41d9-9e92-2caad2fa63cf req-7f9613b9-cf78-4890-80b2-26d67b913728 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 1098 time: 0.0843389 May 31 11:55:05.582380 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:55:05.588439 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device d447eadc-67c7-4eca-8295-e5ae670ce616 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:55:05.599021 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c3c5d246-6a37-41d9-9e92-2caad2fa63cf req-84983f2e-748c-488b-a279-dc067bde970f service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0446143 May 31 11:55:05.731704 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad86fb59-152a-45e9-a39d-cfd88cec60d5 req-b5b38c1c-dcea-4729-b76c-2e94cab33f5a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:05.735763 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ad86fb59-152a-45e9-a39d-cfd88cec60d5 req-b5b38c1c-dcea-4729-b76c-2e94cab33f5a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1392617 May 31 11:55:05.803710 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ad86fb59-152a-45e9-a39d-cfd88cec60d5 req-fd51d0f5-eb24-414f-9506-e25906f77a2b tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0591328 May 31 11:55:05.913236 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:05.928585 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'port_id': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'mac_address': 'fa:16:3e:5a:6f:46', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 371, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:55:06.004405 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port d447eadc-67c7-4eca-8295-e5ae670ce616 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:55:06.013110 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port d447eadc-67c7-4eca-8295-e5ae670ce616 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:55:06.013796 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:06.066537 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1879949-d2ca-4b5d-afcd-8f5915d55f03 req-aa82c46e-2a14-4c38-8514-683c0b15c0eb tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:06.068926 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e1879949-d2ca-4b5d-afcd-8f5915d55f03 req-aa82c46e-2a14-4c38-8514-683c0b15c0eb tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1293726 May 31 11:55:06.075175 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:06.114244 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id e0a8cb19-42fc-4a14-b86a-efda3d732b98 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:55:06.154834 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:06.169073 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e1879949-d2ca-4b5d-afcd-8f5915d55f03 req-42fd4031-f191-4b5f-bd16-7b088bcff205 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0933206 May 31 11:55:06.242120 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:06.248088 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:06.304846 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:06.439201 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:06.443506 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port d447eadc-67c7-4eca-8295-e5ae670ce616, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=d447eadc-67c7-4eca-8295-e5ae670ce616,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:06.444644 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:06.486072 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-99d8bd47-dc61-4a50-b0e8-1ecbef8cd793 req-4fc0fb9a-fe4b-4952-9bea-44bec0456387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:06.487546 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-99d8bd47-dc61-4a50-b0e8-1ecbef8cd793 req-4fc0fb9a-fe4b-4952-9bea-44bec0456387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 944 time: 0.1097503 May 31 11:55:06.536420 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-99d8bd47-dc61-4a50-b0e8-1ecbef8cd793 req-f53bedc9-6f55-41a7-b2c5-ea7b1889dc35 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0429797 May 31 11:55:06.619651 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:06.621281 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:06.656355 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['e0a8cb19-42fc-4a14-b86a-efda3d732b98'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:55:06.746003 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=d447eadc-67c7-4eca-8295-e5ae670ce616,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:06.749834 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:06.920022 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'd447eadc-67c7-4eca-8295-e5ae670ce616'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:06.922358 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-736a5452-532c-4219-bfed-a4517339d40a" -d '{"events": [{"server_uuid": "f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6", "name": "network-vif-plugged", "status": "completed", "tag": "d447eadc-67c7-4eca-8295-e5ae670ce616"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:07.007475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b42fae97-2b6f-4c55-a21e-cb2a8f783811 req-e17b5bd4-5850-4861-ae2f-ae95c8fb0012 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:07.009463 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b42fae97-2b6f-4c55-a21e-cb2a8f783811 req-e17b5bd4-5850-4861-ae2f-ae95c8fb0012 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0910282 May 31 11:55:07.093240 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b42fae97-2b6f-4c55-a21e-cb2a8f783811 req-c7c45ec1-3bff-43c2-a65d-54d2da9a91f2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0768256 May 31 11:55:07.155255 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:55:06 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-aecdd77c-c442-414d-a223-724675604abd x-openstack-request-id: req-aecdd77c-c442-414d-a223-724675604abd {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:07.155787 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6", "name": "network-vif-plugged", "status": "completed", "tag": "d447eadc-67c7-4eca-8295-e5ae670ce616", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:07.155787 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-aecdd77c-c442-414d-a223-724675604abd {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:07.156194 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'code': 200} May 31 11:55:07.421162 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec224ee7-c4c9-4af3-9167-ba09ad46a015 req-ef473a98-fa1e-4aa3-aedd-05f40c76f112 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:07.422779 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ec224ee7-c4c9-4af3-9167-ba09ad46a015 req-ef473a98-fa1e-4aa3-aedd-05f40c76f112 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 944 time: 0.0939703 May 31 11:55:07.555349 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ec224ee7-c4c9-4af3-9167-ba09ad46a015 req-27a731f2-171e-4664-a23f-1e4036b1c87c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1231172 May 31 11:55:07.596911 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:55:07.800969 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd8a9509-f6eb-44c3-a09f-c4c2bb18d0ba req-e49c8f32-f95d-44b3-81a9-4f11d786d261 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:07.802469 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-dd8a9509-f6eb-44c3-a09f-c4c2bb18d0ba req-e49c8f32-f95d-44b3-81a9-4f11d786d261 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1057849 May 31 11:55:07.804353 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05ae06da-605a-416f-bfbc-7173d7bcea61 req-1b0f574d-d738-4615-8241-179f3f90a6e1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:07.805737 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-05ae06da-605a-416f-bfbc-7173d7bcea61 req-1b0f574d-d738-4615-8241-179f3f90a6e1 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 1098 time: 0.1442091 May 31 11:55:07.874694 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-dd8a9509-f6eb-44c3-a09f-c4c2bb18d0ba req-07a7b2f8-f167-49c4-be12-10fed77e2d2c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0653076 May 31 11:55:07.878036 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-05ae06da-605a-416f-bfbc-7173d7bcea61 req-22b8d866-eb8d-43e5-bdc6-014cf55647f8 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0555153 May 31 11:55:07.882930 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port e0a8cb19-42fc-4a14-b86a-efda3d732b98 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:55:07.888791 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port e0a8cb19-42fc-4a14-b86a-efda3d732b98 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:55:07.889161 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:08.009581 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b4b013d-ab66-467f-a7df-7e81b8541a01 req-73e5605c-e179-4cff-858a-67a13d06640a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:08.011788 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7b4b013d-ab66-467f-a7df-7e81b8541a01 req-73e5605c-e179-4cff-858a-67a13d06640a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1256011 May 31 11:55:08.030852 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:08.055528 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-b204d827-5124-4987-b8f3-ff2052449e23 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:08.057173 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-b204d827-5124-4987-b8f3-ff2052449e23 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 1098 time: 0.0847154 May 31 11:55:08.072457 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7b4b013d-ab66-467f-a7df-7e81b8541a01 req-c2a9feca-176a-4bf7-8846-5e9ab684db4f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0544755 May 31 11:55:08.072701 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:08.202041 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-81c2e396-c4be-42d4-a778-17ad4e7e15fa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:08.203466 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-81c2e396-c4be-42d4-a778-17ad4e7e15fa service neutron] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 920 time: 0.1394687 May 31 11:55:08.247666 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:08.252954 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:08.278289 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-612832b6-f439-430a-a778-c49f5fa97539 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e HTTP/1.1" status: 200 len: 217 time: 0.0669608 May 31 11:55:08.336077 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:08.344225 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:08.351122 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-bfd62900-07cf-47a7-af56-ef031bd61097 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:08.352532 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-bfd62900-07cf-47a7-af56-ef031bd61097 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.0653856 May 31 11:55:08.402737 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-ef89980d-7295-46ff-90e7-f7e4d01bb46d service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0417218 May 31 11:55:08.554576 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-ebbf3152-cda7-41e9-ac24-429ae4798c18 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1439378 May 31 11:55:08.754459 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c5113d-63be-4060-88d9-5ca26f3f9870 req-f7e610ab-2e19-46b0-9d06-e47afa984566 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1876414 May 31 11:55:08.781200 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57fb0ced-569b-4432-97be-5af832f0553a req-eaf2fedb-86c2-490e-bd20-d95246253fc4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:08.782671 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-57fb0ced-569b-4432-97be-5af832f0553a req-eaf2fedb-86c2-490e-bd20-d95246253fc4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0995104 May 31 11:55:08.831260 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-57fb0ced-569b-4432-97be-5af832f0553a req-9f6c2218-3b29-4d27-9dad-33d91b2781e1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0427637 May 31 11:55:09.037463 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:09.074698 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b50eff5e-7da6-47a9-b9ae-f75327482521 req-4f9c4516-21b3-4506-bad0-86a843902ef6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:09.076264 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b50eff5e-7da6-47a9-b9ae-f75327482521 req-4f9c4516-21b3-4506-bad0-86a843902ef6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0868387 May 31 11:55:09.121331 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b50eff5e-7da6-47a9-b9ae-f75327482521 req-6382323a-3a2f-4554-8b1e-623104b94a0f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393691 May 31 11:55:09.157515 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:09.160039 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-529aaf4a-4fda-40d5-9cf0-445528f08318" -d '{"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-plugged", "status": "completed", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:09.207030 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:55:09 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-14dfc5af-08b8-4335-bb10-5119e5605a33 x-openstack-request-id: req-14dfc5af-08b8-4335-bb10-5119e5605a33 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:09.207506 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-plugged", "status": "completed", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:09.207856 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-14dfc5af-08b8-4335-bb10-5119e5605a33 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:09.208370 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'code': 200} May 31 11:55:09.299750 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-170c5424-02de-4df9-938b-f32a46ba4ade req-5f88259e-10a4-4a8d-b6a4-4b596936cef7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:09.301192 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-170c5424-02de-4df9-938b-f32a46ba4ade req-5f88259e-10a4-4a8d-b6a4-4b596936cef7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0869024 May 31 11:55:09.348003 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-170c5424-02de-4df9-938b-f32a46ba4ade req-5bdf277e-6dbb-43f5-bb26-1d95ff2d583a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404000 May 31 11:55:09.702586 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0101708e-df43-4d30-9ccd-8aa3d2950bfd req-a25ebc7d-b6fd-45e8-b525-a63c5a4d9431 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:09.705055 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0101708e-df43-4d30-9ccd-8aa3d2950bfd req-a25ebc7d-b6fd-45e8-b525-a63c5a4d9431 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 1098 time: 0.0921137 May 31 11:55:09.769040 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0101708e-df43-4d30-9ccd-8aa3d2950bfd req-e6357659-dbb1-4b4a-a46a-bce31139c728 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0461063 May 31 11:55:10.104254 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30ccd635-813c-4c2a-bbaa-d2f609f9c545 req-63e077e8-9a89-4fa1-af8f-3e43bfd9985b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:10.105741 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-30ccd635-813c-4c2a-bbaa-d2f609f9c545 req-63e077e8-9a89-4fa1-af8f-3e43bfd9985b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1537099 May 31 11:55:10.155550 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-30ccd635-813c-4c2a-bbaa-d2f609f9c545 req-50992d8c-0161-4225-97ea-9b75402be288 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0438850 May 31 11:55:10.310194 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85b0a6b4-0bbc-4f21-824e-3ad0ddd228b7 req-8180a053-cf83-4a00-8082-68c58cdc3edb tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:10.311538 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-85b0a6b4-0bbc-4f21-824e-3ad0ddd228b7 req-8180a053-cf83-4a00-8082-68c58cdc3edb tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0797336 May 31 11:55:10.358434 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-85b0a6b4-0bbc-4f21-824e-3ad0ddd228b7 req-d1a88570-fe38-45ab-928c-44d074fad6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0408120 May 31 11:55:10.552506 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2d413bd1-8854-4568-97d6-ba9e8693d6aa req-20bde6b4-6ce4-4477-a531-909aef28b97c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:10.554183 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2d413bd1-8854-4568-97d6-ba9e8693d6aa req-20bde6b4-6ce4-4477-a531-909aef28b97c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0937669 May 31 11:55:10.564589 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58506) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:10.572474 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'floatingip': {'port_id': 'd447eadc-67c7-4eca-8295-e5ae670ce616'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:55:10.647068 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca01524b-97cc-46c9-b3be-6eec062ff51d req-cd8956cb-b989-49ec-af60-865c6a7f383d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:10.650240 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca01524b-97cc-46c9-b3be-6eec062ff51d req-cd8956cb-b989-49ec-af60-865c6a7f383d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1836555 May 31 11:55:10.687859 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:10.825902 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca01524b-97cc-46c9-b3be-6eec062ff51d req-879d76e9-7477-4441-8d87-5ebf3c73b3d3 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1651080 May 31 11:55:11.367314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-105cf6fd-8852-4127-979d-795b174fa6bc req-9c236b7c-da59-4f81-9ab7-60e69e25a966 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:11.369230 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-105cf6fd-8852-4127-979d-795b174fa6bc req-9c236b7c-da59-4f81-9ab7-60e69e25a966 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0891571 May 31 11:55:11.455674 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-105cf6fd-8852-4127-979d-795b174fa6bc req-c3b7a1f6-5457-4ae3-b997-df39a1d5e335 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0779552 May 31 11:55:11.535171 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:11.601394 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:11.991436 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Floating IP 271403e9-87b5-4d80-b8da-95aafb8d1875 associated. External IP: 172.27.1.102, port: d447eadc-67c7-4eca-8295-e5ae670ce616. May 31 11:55:12.182922 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8fb20718-d848-426f-823d-528133b8edea req-504a285d-63c8-4a53-9de9-7c919224713f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:12.184290 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8fb20718-d848-426f-823d-528133b8edea req-504a285d-63c8-4a53-9de9-7c919224713f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.2401972 May 31 11:55:12.307974 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8fb20718-d848-426f-823d-528133b8edea req-3732aeab-d61c-434d-9d61-51ec913762c7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1169350 May 31 11:55:12.409167 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Router 6c5aae18-4faa-4947-af35-4078d428c2b9 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:55:12.599011 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:55:12.617483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:12.621082 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] Sync routers for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:55:12.630925 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:55:12.631564 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:12.682962 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d8ecf52-49f9-41db-a632-68b64b058e2e req-10034079-be09-4e71-b4c0-468d76243f4d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:12.684438 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6d8ecf52-49f9-41db-a632-68b64b058e2e req-10034079-be09-4e71-b4c0-468d76243f4d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1049664 May 31 11:55:12.685340 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-6e7191ed-6eca-4e9e-9b1f-c563d495dc38 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "PUT /v2.0/floatingips/271403e9-87b5-4d80-b8da-95aafb8d1875 HTTP/1.1" status: 200 len: 991 time: 2.1196699 May 31 11:55:12.686253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'tag': 'd447eadc-67c7-4eca-8295-e5ae670ce616'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:12.688823 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91" -d '{"events": [{"name": "network-changed", "server_uuid": "f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6", "tag": "d447eadc-67c7-4eca-8295-e5ae670ce616"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:12.766859 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6d8ecf52-49f9-41db-a632-68b64b058e2e req-76e53087-bc91-4f46-892b-293ddd8fb325 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0590549 May 31 11:55:12.822852 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:55:12 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-a51b4af3-64a8-431e-9957-296d36e82a70 x-openstack-request-id: req-a51b4af3-64a8-431e-9957-296d36e82a70 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:12.823268 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6", "tag": "d447eadc-67c7-4eca-8295-e5ae670ce616", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:12.823752 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-a51b4af3-64a8-431e-9957-296d36e82a70 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:12.824238 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'tag': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'status': 'completed', 'code': 200} May 31 11:55:12.909277 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:55:12.995687 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-bb2d3156-c7e0-4d0a-92c6-773c7ad58df5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:12.997433 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-bb2d3156-c7e0-4d0a-92c6-773c7ad58df5 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 1098 time: 0.1423266 May 31 11:55:13.259548 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-0cf24e17-4672-4cf2-aba3-c87fb6ae248c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:13.260870 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-0cf24e17-4672-4cf2-aba3-c87fb6ae248c service neutron] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 912 time: 0.1577871 May 31 11:55:13.268314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ab4da2b1-dfab-47db-985c-1b9a193ba6d1 req-8ce465b5-0e53-4770-99cf-9bf47c45836b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:13.269666 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ab4da2b1-dfab-47db-985c-1b9a193ba6d1 req-8ce465b5-0e53-4770-99cf-9bf47c45836b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.2713454 May 31 11:55:13.280382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58574) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:13.286081 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'floatingip': {'port_id': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:55:13.324532 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '271403e9-87b5-4d80-b8da-95aafb8d1875', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'floating_ip_address': '172.27.1.102', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'port_id': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 135, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'mac_address': 'fa:16:3e:3a:c0:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:54:42Z', 'updated_at': '2021-05-31T11:55:11Z', 'revision_number': 1, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:13.329815 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] Checking router: 6c5aae18-4faa-4947-af35-4078d428c2b9 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:55:13.339635 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:55:13.340293 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:13.369552 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-66b0b7cd-713f-48d3-86ba-0385b5074dcc service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=d447eadc-67c7-4eca-8295-e5ae670ce616 HTTP/1.1" status: 200 len: 999 time: 0.0983441 May 31 11:55:13.397636 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:13.448315 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-9e13c3a5-218e-4cc0-b7b3-934d0573b8df service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:13.450243 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-9e13c3a5-218e-4cc0-b7b3-934d0573b8df service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0690086 May 31 11:55:13.540264 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-b7f80d10-438e-46d4-97e7-c45a052bc175 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:13.541835 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-b7f80d10-438e-46d4-97e7-c45a052bc175 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0839393 May 31 11:55:13.569428 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:55:13.704412 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-690100b2-6e67-4dbf-8149-aaafd31b24d0 req-ff7b6a6d-47ba-4219-af4b-3f24a54134cf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:13.705920 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-690100b2-6e67-4dbf-8149-aaafd31b24d0 req-ff7b6a6d-47ba-4219-af4b-3f24a54134cf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.2287123 May 31 11:55:13.743749 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-50cc147a-3e6f-4a52-a28e-840c5714f1c4 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1840727 May 31 11:55:13.913829 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b4a34d56-0ca5-4b95-bfd3-0550fa7d1e91 req-a600f7f3-bfd2-4b10-8a1e-c81bb139d57e service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1631711 May 31 11:55:13.969767 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-690100b2-6e67-4dbf-8149-aaafd31b24d0 req-145c3a03-dbc6-4481-aa71-01e41c4e3a52 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1885209 May 31 11:55:14.040897 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '271403e9-87b5-4d80-b8da-95aafb8d1875', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'floating_ip_address': '172.27.1.102', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'port_id': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 135, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'mac_address': 'fa:16:3e:3a:c0:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:54:42Z', 'updated_at': '2021-05-31T11:55:11Z', 'revision_number': 1, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:14.054790 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2641e3c7-26b8-42b7-9598-cf3a0f499058 None None] The sync data for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi is: [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '271403e9-87b5-4d80-b8da-95aafb8d1875', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'floating_ip_address': '172.27.1.102', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'port_id': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 135, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'mac_address': 'fa:16:3e:3a:c0:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:54:42Z', 'updated_at': '2021-05-31T11:55:11Z', 'revision_number': 1, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:55:14.299941 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-282ce180-999f-4a29-92bb-505018c21802 None None] New status for floating IP 271403e9-87b5-4d80-b8da-95aafb8d1875: ACTIVE {{(pid=99041) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 11:55:14.394749 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:15.162549 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-58688073-00a8-4385-840f-1149967c9803 req-5994ed6b-0910-4d7e-a495-675e38e644b2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:15.163996 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-58688073-00a8-4385-840f-1149967c9803 req-5994ed6b-0910-4d7e-a495-675e38e644b2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0914536 May 31 11:55:15.214242 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-58688073-00a8-4385-840f-1149967c9803 req-6a0b3e5d-1f33-4bdd-9428-f7112d33eb14 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0435753 May 31 11:55:15.309802 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:15.494852 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Floating IP abe27c62-2687-4502-83b6-bd3a48c06f27 associated. External IP: 172.27.1.172, port: e0a8cb19-42fc-4a14-b86a-efda3d732b98. May 31 11:55:15.970672 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:55:16.184605 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:55:16.193071 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:16.195564 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] Sync routers for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:55:16.205724 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:55:16.206451 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:16.263367 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-7945576d-71b7-4d8a-bad8-1c80f1f22140 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "PUT /v2.0/floatingips/abe27c62-2687-4502-83b6-bd3a48c06f27 HTTP/1.1" status: 200 len: 991 time: 2.9734509 May 31 11:55:16.263367 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:16.263367 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3" -d '{"events": [{"name": "network-changed", "server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:16.429460 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:55:16.439553 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5492ffab-0bd0-48dc-9c90-f60a57cf492e req-f9faae7b-df5e-4682-af93-0b1de6335d50 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:16.440998 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5492ffab-0bd0-48dc-9c90-f60a57cf492e req-f9faae7b-df5e-4682-af93-0b1de6335d50 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0828648 May 31 11:55:16.492147 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:55:16 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-c1db9a11-9d76-4ddb-a782-1faa61c84276 x-openstack-request-id: req-c1db9a11-9d76-4ddb-a782-1faa61c84276 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:16.493722 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:16.494352 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-c1db9a11-9d76-4ddb-a782-1faa61c84276 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:16.495339 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'status': 'completed', 'code': 200} May 31 11:55:16.503236 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5492ffab-0bd0-48dc-9c90-f60a57cf492e req-09cc518a-abec-4740-b121-524497f9d4cf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0518770 May 31 11:55:16.644065 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-5b3ef199-0d86-43b1-873c-6006f23f6488 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:16.645624 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-5b3ef199-0d86-43b1-873c-6006f23f6488 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 1098 time: 0.1095479 May 31 11:55:16.667867 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:55:16.842427 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'abe27c62-2687-4502-83b6-bd3a48c06f27', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'floating_ip_address': '172.27.1.172', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'port_id': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 137, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'mac_address': 'fa:16:3e:5a:6f:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:54:44Z', 'updated_at': '2021-05-31T11:55:14Z', 'revision_number': 1, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:16.847026 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] Checking router: 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:55:16.863986 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:55:16.864666 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:54:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:16.864784 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1ea3154-05d9-4ee3-af1f-9703fc7b04e2 req-b0c8c0cc-67cb-450e-bec2-753a5191a253 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:16.866164 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f1ea3154-05d9-4ee3-af1f-9703fc7b04e2 req-b0c8c0cc-67cb-450e-bec2-753a5191a253 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 945 time: 0.2078569 May 31 11:55:16.871543 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-7adb69c5-1503-4823-a658-1b63dacba006 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:16.872877 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-7adb69c5-1503-4823-a658-1b63dacba006 service neutron] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 917 time: 0.1747813 May 31 11:55:16.999795 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f1ea3154-05d9-4ee3-af1f-9703fc7b04e2 req-5e8e3f62-5914-4aea-898f-6799ff98464e tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1259177 May 31 11:55:17.006328 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-6210851b-3b0d-4f7f-bd91-eed8132c4706 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98 HTTP/1.1" status: 200 len: 999 time: 0.1194756 May 31 11:55:17.084729 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-e8014b7a-0f5c-428f-9a73-135663573204 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:17.086104 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-e8014b7a-0f5c-428f-9a73-135663573204 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0630257 May 31 11:55:17.114752 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:55:17.165750 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-4ce8eba9-ed1e-47bc-adac-7a6eae288d23 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:17.167272 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-4ce8eba9-ed1e-47bc-adac-7a6eae288d23 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0730436 May 31 11:55:17.320292 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-b6201760-a84c-469c-854b-a2d34d88a60e service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1341202 May 31 11:55:17.349084 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58654) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:17.490744 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:17.507768 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:17.512349 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Usage tracker for resource:security_group_rule and tenant:be7757c993194411ac5dbffcacfd612a is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:17.516641 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:17.543009 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:17.547941 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd06a714-93ac-48eb-aa5d-59a4bd14d1b3 req-2977bd45-4e37-4f35-bc61-eb08321e403b service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2188866 May 31 11:55:17.554155 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'abe27c62-2687-4502-83b6-bd3a48c06f27', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'floating_ip_address': '172.27.1.172', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'port_id': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 137, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'mac_address': 'fa:16:3e:5a:6f:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:54:44Z', 'updated_at': '2021-05-31T11:55:14Z', 'revision_number': 1, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:17.558510 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4d8b93fa-81c3-4183-9e04-ce6a41c27dec None None] The sync data for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi is: [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'abe27c62-2687-4502-83b6-bd3a48c06f27', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'floating_ip_address': '172.27.1.172', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'port_id': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 137, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'mac_address': 'fa:16:3e:5a:6f:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:54:44Z', 'updated_at': '2021-05-31T11:55:14Z', 'revision_number': 1, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:55:17.643169 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:17.738436 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:17.746618 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8fa7e602-a8ef-4315-8d18-ad9209889ee1 req-ffccdfc8-959f-4f20-99e6-33beb86c8886 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:17.748065 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8fa7e602-a8ef-4315-8d18-ad9209889ee1 req-ffccdfc8-959f-4f20-99e6-33beb86c8886 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1065729 May 31 11:55:17.748414 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-de6c7a20-baac-40f2-9e3e-cd8d52d0b6d7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:17.792207 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=be7757c993194411ac5dbffcacfd612a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4416754 May 31 11:55:17.797835 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58668) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:17.803753 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8fa7e602-a8ef-4315-8d18-ad9209889ee1 req-f5698df0-7361-45a1-946d-39dbb02e578d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0491836 May 31 11:55:17.819566 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0f9c2717-c8fb-4b06-84f0-dcda8b48833d,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:17.843755 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.843755 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.843755 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.843755 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.843755 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.844303 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-09bca162-b54c-4521-9034-a0239b2c4098 None None] New status for floating IP abe27c62-2687-4502-83b6-bd3a48c06f27: ACTIVE {{(pid=99041) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 11:55:17.844686 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.844686 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.844686 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.844686 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.844686 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.844686 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.844686 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:17.844686 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:17.880342 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1d84698e-26b7-4b1a-afff-b10a8b11359d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:17.908203 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1d84698e-26b7-4b1a-afff-b10a8b11359d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:18.109873 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1d84698e-26b7-4b1a-afff-b10a8b11359d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:18.114774 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:18.162402 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1d84698e-26b7-4b1a-afff-b10a8b11359d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0f9c2717-c8fb-4b06-84f0-dcda8b48833d,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:18.164738 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1d84698e-26b7-4b1a-afff-b10a8b11359d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:18.182916 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1d84698e-26b7-4b1a-afff-b10a8b11359d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:18.186744 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-1d84698e-26b7-4b1a-afff-b10a8b11359d admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/0f9c2717-c8fb-4b06-84f0-dcda8b48833d HTTP/1.1" status: 204 len: 168 time: 0.3867340 May 31 11:55:18.216934 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-450301f5-7ae0-4e78-8168-4e0572204775 req-bd2227d5-4cee-4216-81b6-bf0b843ba3e5 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:18.218398 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-450301f5-7ae0-4e78-8168-4e0572204775 req-bd2227d5-4cee-4216-81b6-bf0b843ba3e5 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 945 time: 0.1045473 May 31 11:55:18.264232 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-450301f5-7ae0-4e78-8168-4e0572204775 req-3e35abe7-afa1-43cc-83b2-72516a6eda08 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0402474 May 31 11:55:18.908136 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb10ffe6-e670-4f78-9849-95817173884a req-454c5c74-c8c0-4e8d-ab24-8fd93973cf8e tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:18.909578 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-cb10ffe6-e670-4f78-9849-95817173884a req-454c5c74-c8c0-4e8d-ab24-8fd93973cf8e tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.47 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 945 time: 0.1007080 May 31 11:55:18.964934 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58692) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:18.982716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.005590 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:19.005590 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Usage tracker for resource:security_group_rule and tenant:b039095d2b8848bdb5a2f58221e60d4a is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:19.006762 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:19.013427 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:19.111555 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b68dfc9-7a37-490f-b273-a3903ccc93f2 req-85d2ab38-84f4-4afa-bcbe-24201eb3cde0 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:19.113014 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5b68dfc9-7a37-490f-b273-a3903ccc93f2 req-85d2ab38-84f4-4afa-bcbe-24201eb3cde0 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1653264 May 31 11:55:19.119084 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.155336 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Deleting port 982ed1f8-4257-4da4-9de6-458b5ec1994d {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:55:19.155793 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.235140 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.238532 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5b68dfc9-7a37-490f-b273-a3903ccc93f2 req-662336a2-4eae-4674-8c66-fd0ce52edc89 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1187081 May 31 11:55:19.245457 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-da73796e-02a9-4df7-b434-9c77b2c48196 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:19.285626 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=b039095d2b8848bdb5a2f58221e60d4a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3193581 May 31 11:55:19.286553 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d7e610e4-3a98-4eff-9e70-a72c67dcbf4f,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:19.288870 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58702) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:19.291501 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:19.311913 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] For port 982ed1f8-4257-4da4-9de6-458b5ec1994d, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=982ed1f8-4257-4da4-9de6-458b5ec1994d,segment=NetworkSegment(f3279336-e17b-42d0-9e25-1cefbd5c68b4),segment_id=f3279336-e17b-42d0-9e25-1cefbd5c68b4)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:19.312264 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.447153 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Calling delete_port for 982ed1f8-4257-4da4-9de6-458b5ec1994d owned by compute:nova {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:55:19.502056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41394f23-aa64-42de-bb0e-87a0e39f9c90 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.534952 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41394f23-aa64-42de-bb0e-87a0e39f9c90 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.560999 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0e15593-7a02-482f-a86b-e883c3d0d64e req-940d8a56-dd49-4c1f-9ad8-fbab38872cdb tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:19.562382 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0e15593-7a02-482f-a86b-e883c3d0d64e req-940d8a56-dd49-4c1f-9ad8-fbab38872cdb tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 945 time: 0.1333437 May 31 11:55:19.621863 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0e15593-7a02-482f-a86b-e883c3d0d64e req-9e3d5c9f-9bd1-46d8-9090-f62e8cd76a03 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/security-groups?id=45609a47-7b41-4914-8efd-807ab69ca2c6&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0466039 May 31 11:55:19.732466 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.739115 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:19.757463 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['982ed1f8-4257-4da4-9de6-458b5ec1994d'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:55:19.785850 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Pushing event deleted for resources: {'Port': ['ID=982ed1f8-4257-4da4-9de6-458b5ec1994d,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:19.795546 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41394f23-aa64-42de-bb0e-87a0e39f9c90 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.804837 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:19.811926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-41394f23-aa64-42de-bb0e-87a0e39f9c90 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d7e610e4-3a98-4eff-9e70-a72c67dcbf4f,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:19.815752 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-41394f23-aa64-42de-bb0e-87a0e39f9c90 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:19.818991 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41394f23-aa64-42de-bb0e-87a0e39f9c90 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.820255 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-41394f23-aa64-42de-bb0e-87a0e39f9c90 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/d7e610e4-3a98-4eff-9e70-a72c67dcbf4f HTTP/1.1" status: 204 len: 168 time: 0.5304852 May 31 11:55:19.821850 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.822202 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.822676 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.822949 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.823336 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.823611 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.823940 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.824274 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.824962 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.824962 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.825225 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.825546 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:19.833232 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:19.876024 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] No DVR routers for this DVR port 982ed1f8-4257-4da4-9de6-458b5ec1994d on host hv-cinder-79123 {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:55:19.878951 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:19.881644 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4', 'name': 'network-vif-deleted', 'tag': '982ed1f8-4257-4da4-9de6-458b5ec1994d'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:19.884662 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-e283a38b-e4be-460c-9417-171d750f33d5" -d '{"events": [{"server_uuid": "d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4", "name": "network-vif-deleted", "tag": "982ed1f8-4257-4da4-9de6-458b5ec1994d"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:19.930638 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:55:19 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-27612498-900c-42e8-bec5-d283438f5bfc x-openstack-request-id: req-27612498-900c-42e8-bec5-d283438f5bfc {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:19.931383 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4", "name": "network-vif-deleted", "tag": "982ed1f8-4257-4da4-9de6-458b5ec1994d", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:19.931383 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-27612498-900c-42e8-bec5-d283438f5bfc {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:19.931747 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4', 'name': 'network-vif-deleted', 'tag': '982ed1f8-4257-4da4-9de6-458b5ec1994d', 'status': 'completed', 'code': 200} May 31 11:55:20.018910 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-cb10ffe6-e670-4f78-9849-95817173884a req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.47 "DELETE /v2.0/ports/982ed1f8-4257-4da4-9de6-458b5ec1994d HTTP/1.1" status: 204 len: 173 time: 1.1036644 May 31 11:55:20.436553 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-91c72664-0cb2-4247-a5d2-e21ed5ddf0e9 req-b3803be3-5833-475a-8e3b-c457df2fed34 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:20.438030 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-91c72664-0cb2-4247-a5d2-e21ed5ddf0e9 req-b3803be3-5833-475a-8e3b-c457df2fed34 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0835395 May 31 11:55:20.483835 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-91c72664-0cb2-4247-a5d2-e21ed5ddf0e9 req-1270ad50-7a9c-4839-ba49-3cdb0d01a701 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0399444 May 31 11:55:20.784034 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8c66fdc9-16cd-44bf-8f14-3a5a6c737988 req-e9e9b18e-df03-494c-bd22-46cc72dcf97f tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.0437818 May 31 11:55:20.852693 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58752) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:20.970585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:20.985695 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:20.990001 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6c8dc9b3-9c1d-440b-8588-caef69cef58b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:20.990483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Usage tracker for resource:security_group_rule and tenant:7ddc59fd2a924a968a8666e448f0d1dc is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:20.994600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:21.000516 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:21.089150 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:21.286528 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:21.299075 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:21.343084 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=7ddc59fd2a924a968a8666e448f0d1dc&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4868546 May 31 11:55:21.343084 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=fb786b2d-8b0d-4de2-9ec8-0049abde805f,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:21.343600 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 58774) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:21.346103 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:21.408908 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7db20818-bb6b-4eaa-a60a-4c3ff1c3ba4d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:21.434852 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7db20818-bb6b-4eaa-a60a-4c3ff1c3ba4d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:21.589197 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7db20818-bb6b-4eaa-a60a-4c3ff1c3ba4d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:21.594206 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:21.643807 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7db20818-bb6b-4eaa-a60a-4c3ff1c3ba4d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=fb786b2d-8b0d-4de2-9ec8-0049abde805f,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:21.651746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7db20818-bb6b-4eaa-a60a-4c3ff1c3ba4d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:21.652826 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7db20818-bb6b-4eaa-a60a-4c3ff1c3ba4d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:21.654061 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-7db20818-bb6b-4eaa-a60a-4c3ff1c3ba4d admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/fb786b2d-8b0d-4de2-9ec8-0049abde805f HTTP/1.1" status: 204 len: 168 time: 0.3092191 May 31 11:55:21.701816 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9ea9977-f52c-4aa7-8548-b7674cd8bf8a req-80cb27c7-c2eb-4435-89cc-dd405e891a2c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:21.703100 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a9ea9977-f52c-4aa7-8548-b7674cd8bf8a req-80cb27c7-c2eb-4435-89cc-dd405e891a2c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0973310 May 31 11:55:21.762480 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a9ea9977-f52c-4aa7-8548-b7674cd8bf8a req-fd983880-2ecb-4919-af7a-d50c6e68a2cc tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0534112 May 31 11:55:21.952035 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-af89965d-cf13-4032-a0ac-a1e215ea36eb req-31208430-719e-4f14-8fd8-793e4c9a5aa2 tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] 10.222.0.48 "GET /v2.0/ports?device_id=d1aa3dbd-a46c-4e2c-9ca2-df7e6a8417c4 HTTP/1.1" status: 200 len: 210 time: 0.0423574 May 31 11:55:22.684537 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 58812) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:22.702532 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:22.717270 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:22.721163 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Usage tracker for resource:security_group_rule and tenant:7c7feeceab9a4079a47f889c9c7597ec is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:22.724022 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:22.739675 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:22.817969 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:22.958700 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f23b30b1-3dc0-4041-8ff0-1435e2a79842 req-189db0cb-0f1c-4a92-8542-caa81d514bef tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:22.960106 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f23b30b1-3dc0-4041-8ff0-1435e2a79842 req-189db0cb-0f1c-4a92-8542-caa81d514bef tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0837343 May 31 11:55:22.982027 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:22.994804 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:23.049406 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=7c7feeceab9a4079a47f889c9c7597ec&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3637197 May 31 11:55:23.052487 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 58820) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:23.053641 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f23b30b1-3dc0-4041-8ff0-1435e2a79842 req-daec8123-416e-410a-9715-9c88d1ef08f1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0878377 May 31 11:55:23.054414 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0258fd77-5ec2-4970-85d5-ddd9986f8fe7,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:23.058090 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:23.128628 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0258e847-60e8-43f1-95d1-9535d1f89785 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:23.155753 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0258e847-60e8-43f1-95d1-9535d1f89785 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:23.332562 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0258e847-60e8-43f1-95d1-9535d1f89785 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:23.338544 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:23.353332 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0258e847-60e8-43f1-95d1-9535d1f89785 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0258fd77-5ec2-4970-85d5-ddd9986f8fe7,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:23.365188 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0258e847-60e8-43f1-95d1-9535d1f89785 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:23.365188 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0258e847-60e8-43f1-95d1-9535d1f89785 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:23.365188 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-0258e847-60e8-43f1-95d1-9535d1f89785 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/0258fd77-5ec2-4970-85d5-ddd9986f8fe7 HTTP/1.1" status: 204 len: 168 time: 0.3070281 May 31 11:55:23.423722 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:23.520575 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-85074769-4adc-437b-897a-57eb0a8d50cc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:23.698768 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 982ed1f8-4257-4da4-9de6-458b5ec1994d no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:55:23.736953 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 982ed1f8-4257-4da4-9de6-458b5ec1994d {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:55:23.737300 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 982ed1f8-4257-4da4-9de6-458b5ec1994d not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:55:23.835807 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-2d8fcba7-ba27-40c1-9cfc-2b52fc42bff6 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:23.837991 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-2d8fcba7-ba27-40c1-9cfc-2b52fc42bff6 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 815 time: 0.1724422 May 31 11:55:23.855020 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-ca6fafdc-6385-4ca0-aa34-174e4c3ebd1f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/quotas/7dea63b6c4a340d7b361fe91ca6b8281 HTTP/1.1" status: 200 len: 375 time: 0.0112841 May 31 11:55:23.948805 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-51151866-38b5-49ff-b60e-d008981702a1 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&fields=id HTTP/1.1" status: 200 len: 255 time: 0.0876629 May 31 11:55:24.083324 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-dca66b88-1927-453b-9bac-b9126841cb13 service neutron] 10.222.0.48 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1275258 May 31 11:55:24.220587 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-1a9f3c5d-19c7-4952-96c0-fda4c9b6ebaa service neutron] 10.222.0.48 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1308894 May 31 11:55:24.346367 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a952a1a-3888-40c3-ba34-9795e9531dac req-0a9910d5-0ae6-42f4-a2d7-3184e4b4cfb2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:24.347861 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4a952a1a-3888-40c3-ba34-9795e9531dac req-0a9910d5-0ae6-42f4-a2d7-3184e4b4cfb2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1590452 May 31 11:55:24.393212 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4a952a1a-3888-40c3-ba34-9795e9531dac req-d71c741e-d01f-4b80-bde3-76022e9d1846 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0388227 May 31 11:55:24.588419 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-31e01893-c368-4028-bde9-758f91932daf req-8ec4c982-6f37-45a6-8536-13094ace0d0d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 210 time: 0.0455127 May 31 11:55:24.595310 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:25.120942 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 58930) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:25.182132 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Deleting subnet 94658dd6-8c5b-487d-9286-e8ccb8e389f4 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 11:55:25.185635 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:25.240069 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:25.247136 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:25.567255 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:25.573092 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:25.589836 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7326dc3-d5c0-4753-a14c-443521083359 req-1ffb73fc-87c4-47da-9621-ec66d2408f51 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:25.591416 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c7326dc3-d5c0-4753-a14c-443521083359 req-1ffb73fc-87c4-47da-9621-ec66d2408f51 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0902710 May 31 11:55:25.610061 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:25.631650 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=94658dd6-8c5b-487d-9286-e8ccb8e389f4,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:25.631650 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:25.631650 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/94658dd6-8c5b-487d-9286-e8ccb8e389f4 HTTP/1.1" status: 204 len: 168 time: 0.5062480 May 31 11:55:25.631650 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 58954) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:25.646247 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c7326dc3-d5c0-4753-a14c-443521083359 req-7875783c-8958-466d-aa8a-8357a8193c83 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0484231 May 31 11:55:25.732433 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2e85a0ef-ab71-48b2-9728-6be010431197 req-d2e07619-dfc1-4e03-9ba7-a1b620b3b43d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 210 time: 0.0429671 May 31 11:55:25.745690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:25.879004 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:25.893999 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > segment before_delete 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.894395 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.895037 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.895401 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > port after_create 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.895719 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > port after_delete 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.896142 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > port after_update 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.896519 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > segment after_delete 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.897386 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.898325 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:25.899493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > subnet after_update 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:26.058438 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:26.058946 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:26.059364 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > subnet before_create 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:26.059767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Subscribe: > subnet before_update 55550000 {{(pid=99039) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:55:26.060187 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['02ae88e7-d3df-4dfe-bee0-364625c3e776']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:26.074352 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'f3279336-e17b-42d0-9e25-1cefbd5c68b4') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:26.074670 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'f3279336-e17b-42d0-9e25-1cefbd5c68b4') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:55:26.087607 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:26.104122 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:26.111221 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Releasing vlan 392 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 11:55:26.111596 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: from cryptography.utils import int_from_bytes May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: from cryptography.utils import int_from_bytes May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 11:55:26.127249 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 11:55:26.143390 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:26.280784 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:26.285036 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:26.300883 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:26.321917 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Pushing event deleted for resources: {'Network': ['ID=02ae88e7-d3df-4dfe-bee0-364625c3e776,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:26.325998 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:26.407383 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] 10.222.0.48 "DELETE /v2.0/networks/02ae88e7-d3df-4dfe-bee0-364625c3e776 HTTP/1.1" status: 204 len: 168 time: 0.7757051 May 31 11:55:26.876734 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d819a489-32e1-4a0a-9285-4bf33aeecdef req-a49e72fb-bead-4f45-b4ce-5edf93fe608f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:26.878423 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d819a489-32e1-4a0a-9285-4bf33aeecdef req-a49e72fb-bead-4f45-b4ce-5edf93fe608f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1090879 May 31 11:55:26.927561 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d819a489-32e1-4a0a-9285-4bf33aeecdef req-d4a77aed-6889-48f4-96d2-5ddcc63cf160 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430143 May 31 11:55:27.017961 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-12a7be8d-df22-41ac-91db-53260994b634 req-75d729b0-0dbd-4057-bc30-4883e17cfe80 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 210 time: 0.0438454 May 31 11:55:27.038107 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 59004) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:27.088893 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-915ad4f9-41a5-43f3-9235-a411be69b49d admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=3b87c0540dea485d8ec60a8628fced11&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0493324 May 31 11:55:27.091627 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59008) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:27.234925 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3d8b2ab8-6f7c-4d0e-9447-efc03a3eabfd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:27.234925 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3d8b2ab8-6f7c-4d0e-9447-efc03a3eabfd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:27.714889 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3d8b2ab8-6f7c-4d0e-9447-efc03a3eabfd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:27.719160 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-2d66a2b5-10e9-4ac3-8420-1e597802895b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:27.733265 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3d8b2ab8-6f7c-4d0e-9447-efc03a3eabfd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=45609a47-7b41-4914-8efd-807ab69ca2c6,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:27.735730 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-3d8b2ab8-6f7c-4d0e-9447-efc03a3eabfd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:27.805650 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3d8b2ab8-6f7c-4d0e-9447-efc03a3eabfd admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:27.807053 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-3d8b2ab8-6f7c-4d0e-9447-efc03a3eabfd admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/45609a47-7b41-4914-8efd-807ab69ca2c6 HTTP/1.1" status: 204 len: 168 time: 0.7144001 May 31 11:55:28.192394 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f59f8133-dfba-449e-9aac-49ebf5555eac req-b21067b5-5e68-44e0-903b-ee6525135f1e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 210 time: 0.0511174 May 31 11:55:28.251716 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-454a485f-e857-4767-9f14-b63ceb5923eb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:28.253505 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-454a485f-e857-4767-9f14-b63ceb5923eb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 815 time: 0.2428327 May 31 11:55:28.289226 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c2b2146-d318-4f34-bdec-88e67485ecb5 req-2110b419-c5c1-43f0-8501-a40435aacd79 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:28.291532 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9c2b2146-d318-4f34-bdec-88e67485ecb5 req-2110b419-c5c1-43f0-8501-a40435aacd79 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1110246 May 31 11:55:28.338292 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9c2b2146-d318-4f34-bdec-88e67485ecb5 req-0f6ad170-08e4-4f6e-80ef-1bbff8b8ebe4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409844 May 31 11:55:31.338262 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Request body: {'port': {'device_id': '85c447cc-29b9-4fd4-a3f6-d7219276b581', 'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'admin_state_up': True, 'tenant_id': '7dea63b6c4a340d7b361fe91ca6b8281'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:55:31.518100 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:31.521817 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5f63bb59-e17e-49c2-82d7-6c3bec52c608 req-edc599b8-20b2-496f-b559-98d05e412ba3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 210 time: 0.0685525 May 31 11:55:31.530834 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:31.612013 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Made reservation on behalf of 7dea63b6c4a340d7b361fe91ca6b8281 for: {'port': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:55:31.616607 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:31.697446 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ded0b45-569d-4430-b196-b2a21aa5d85f req-fb6c9764-9b25-4b61-9e52-af70e2bc89a6 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:31.699600 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1ded0b45-569d-4430-b196-b2a21aa5d85f req-fb6c9764-9b25-4b61-9e52-af70e2bc89a6 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1117413 May 31 11:55:31.759972 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1ded0b45-569d-4430-b196-b2a21aa5d85f req-875e0326-afb4-4923-8179-6fe045457d40 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0541670 May 31 11:55:31.939549 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Allocated IP 172.20.0.14 (709626b6-807b-4256-9031-1578948f1fe8/4c58bc8a-d411-461e-a483-39773ff0d6f4/064a6ea1-ceec-462e-b73a-4e2c88f7a5b8) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:55:32.144440 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:32.292643 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:32.298306 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-ab9884b5-8449-4869-b02c-7b5a49daa41c tempest-DeleteServersTestJSON-1483167614 tempest-DeleteServersTestJSON-1483167614-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:32.304322 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['c4a233f2-9446-42a7-964d-1dc2496c938a'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:55:32.369492 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Network 709626b6-807b-4256-9031-1578948f1fe8 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:55:32.373289 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:55:32.377786 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:55:32.436479 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Pushing event updated for resources: {'Port': ['ID=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:32.441316 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.143s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:32.465093 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 59118) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:32.480414 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:32.481082 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:32.617374 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:32.619017 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.2853787 May 31 11:55:32.632392 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Request body: {'port': {'device_id': '85c447cc-29b9-4fd4-a3f6-d7219276b581', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:55:32.634689 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:32.639060 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Usage tracker for resource:security_group_rule and tenant:deca552033f542cf8d729d5385da3cba is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:32.642269 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:32.649323 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:32.743704 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:32.826413 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:32.885288 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-484e3872-9498-4dc0-9e0c-ade38fbb9d54 req-5c9e47dd-447b-4bb1-afe7-1af8b7f5632b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:32.885999 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-484e3872-9498-4dc0-9e0c-ade38fbb9d54 req-5c9e47dd-447b-4bb1-afe7-1af8b7f5632b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:55:32.904185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:32.965261 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c76e2dcd-c6cb-4d9f-afb8-b18902b7fce0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:33.021614 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=44aa1297-21d6-4f0d-baae-4f002be227a0,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:33.027569 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:33.032612 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=deca552033f542cf8d729d5385da3cba&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5635552 May 31 11:55:33.033405 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 59138) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:33.105226 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-484e3872-9498-4dc0-9e0c-ade38fbb9d54 req-5c9e47dd-447b-4bb1-afe7-1af8b7f5632b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:55:33.106655 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-484e3872-9498-4dc0-9e0c-ade38fbb9d54 req-5c9e47dd-447b-4bb1-afe7-1af8b7f5632b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 932 time: 0.4197929 May 31 11:55:33.164925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4646e3b2-4b5e-4eed-bd7f-06c677b900fd req-2e4c33d2-7397-4648-9506-2c2e81e11f6f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:33.166516 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4646e3b2-4b5e-4eed-bd7f-06c677b900fd req-2e4c33d2-7397-4648-9506-2c2e81e11f6f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.2532396 May 31 11:55:33.190690 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-484e3872-9498-4dc0-9e0c-ade38fbb9d54 req-016889f8-9fbe-41c0-8e07-a4f2930e1f2c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0777619 May 31 11:55:33.193761 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-238f928d-3e0e-4ea0-bb4a-89f89b4774dd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.258913 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-238f928d-3e0e-4ea0-bb4a-89f89b4774dd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.268741 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4646e3b2-4b5e-4eed-bd7f-06c677b900fd req-1d7dbd71-cd59-4237-b3f5-c85e47de400c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0804026 May 31 11:55:33.308532 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 {{(pid=99039) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:55:33.315845 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.394648 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-238f928d-3e0e-4ea0-bb4a-89f89b4774dd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.394949 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Transition to ACTIVE for port object 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 will not be triggered until provisioned by entity L2. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:55:33.400789 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-304a9669-5f9a-4234-95ac-0056252d6336 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:33.413548 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-238f928d-3e0e-4ea0-bb4a-89f89b4774dd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=44aa1297-21d6-4f0d-baae-4f002be227a0,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:33.416591 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-238f928d-3e0e-4ea0-bb4a-89f89b4774dd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:33.417533 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-238f928d-3e0e-4ea0-bb4a-89f89b4774dd admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.419651 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-238f928d-3e0e-4ea0-bb4a-89f89b4774dd admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/44aa1297-21d6-4f0d-baae-4f002be227a0 HTTP/1.1" status: 204 len: 168 time: 0.3843303 May 31 11:55:33.451816 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.462891 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 59150) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:33.463060 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:33.614190 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.632798 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:33.638490 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Usage tracker for resource:security_group_rule and tenant:ab9f0ac67842433c9de2f6ae7166bd54 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:33.643333 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:33.651701 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:33.662649 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Pushing event updated for resources: {'Port': ['ID=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:33.669532 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.207s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:33.717800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Attempting to bind port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:55:33.718147 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Attempting to bind port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:55:33.718492 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Attempting to bind port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 on network 709626b6-807b-4256-9031-1578948f1fe8 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:55:33.723525 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 on network 709626b6-807b-4256-9031-1578948f1fe8 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:55:33.724113 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Attempting to bind port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 on network 709626b6-807b-4256-9031-1578948f1fe8 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:55:33.737290 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 55, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:55:33.737290 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Checking segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:55:33.737690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Bound using segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:55:33.737833 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Bound port: 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'}}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:55:33.738181 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.779138 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.851458 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] For port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8, host hv-cinder-79123, cleared binding levels {{(pid=99039) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:55:33.854534 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] For port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99039) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:55:33.874589 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.883866 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-238f928d-3e0e-4ea0-bb4a-89f89b4774dd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:33.927050 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=ab9f0ac67842433c9de2f6ae7166bd54&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4649587 May 31 11:55:33.927050 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=92c4b356-18e5-4da6-aa63-57fd59bbd7e4,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:33.927677 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59164) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:33.928460 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:33.958458 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:33.969922 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:34.048781 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:34.049385 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:34.050909 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] 10.222.0.47 "PUT /v2.0/ports/064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 HTTP/1.1" status: 200 len: 1094 time: 1.4234977 May 31 11:55:34.051386 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '85c447cc-29b9-4fd4-a3f6-d7219276b581', 'tag': '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:34.053730 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-9023a813-d5f4-494c-b81d-07b0d36fdb71" -d '{"events": [{"name": "network-changed", "server_uuid": "85c447cc-29b9-4fd4-a3f6-d7219276b581", "tag": "064a6ea1-ceec-462e-b73a-4e2c88f7a5b8"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:34.097592 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-9f177693-20d7-499e-b996-1f5eb9c10c4b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:34.103026 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:55:34 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-daff6910-6002-425c-bc12-27f7df8005dc x-openstack-request-id: req-daff6910-6002-425c-bc12-27f7df8005dc {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:34.103331 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "85c447cc-29b9-4fd4-a3f6-d7219276b581", "tag": "064a6ea1-ceec-462e-b73a-4e2c88f7a5b8", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:34.103629 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-daff6910-6002-425c-bc12-27f7df8005dc {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:34.104114 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '85c447cc-29b9-4fd4-a3f6-d7219276b581', 'tag': '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8', 'status': 'completed', 'code': 200} May 31 11:55:34.111524 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Pushing event updated for resources: {'Port': ['ID=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,revision_number=3']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:34.117319 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:34.135328 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-9f177693-20d7-499e-b996-1f5eb9c10c4b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:34.238054 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9023a813-d5f4-494c-b81d-07b0d36fdb71 req-61dad577-9423-48ab-9778-85728715fb5d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:34.239501 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9023a813-d5f4-494c-b81d-07b0d36fdb71 req-61dad577-9423-48ab-9778-85728715fb5d service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 1097 time: 0.1068687 May 31 11:55:34.420144 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3db8520a-0c0d-4209-999e-60676c901f98 req-6ae76c5f-a64c-45e8-8d96-f75e6df81331 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:34.421889 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3db8520a-0c0d-4209-999e-60676c901f98 req-6ae76c5f-a64c-45e8-8d96-f75e6df81331 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 944 time: 0.1054025 May 31 11:55:34.423533 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-9f177693-20d7-499e-b996-1f5eb9c10c4b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:34.472463 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e0692338-ebe6-46b3-9022-fe0617b3ae7f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:34.478460 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3db8520a-0c0d-4209-999e-60676c901f98 req-193247a2-00b3-4f2b-bce4-a6d8b39a46f0 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0506301 May 31 11:55:34.511030 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f177693-20d7-499e-b996-1f5eb9c10c4b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=92c4b356-18e5-4da6-aa63-57fd59bbd7e4,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:34.522905 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-9f177693-20d7-499e-b996-1f5eb9c10c4b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:34.552337 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-9f177693-20d7-499e-b996-1f5eb9c10c4b admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:34.553422 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-9f177693-20d7-499e-b996-1f5eb9c10c4b admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/92c4b356-18e5-4da6-aa63-57fd59bbd7e4 HTTP/1.1" status: 204 len: 168 time: 0.6242249 May 31 11:55:34.563683 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9023a813-d5f4-494c-b81d-07b0d36fdb71 req-ba985626-6c17-48ec-b6ef-91c4ba99fd23 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:34.565081 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9023a813-d5f4-494c-b81d-07b0d36fdb71 req-ba985626-6c17-48ec-b6ef-91c4ba99fd23 service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&shared=False HTTP/1.1" status: 200 len: 916 time: 0.3173583 May 31 11:55:34.597937 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-89e35c1a-0be6-461c-8eb2-c1e5e545914a req-4c4bfa55-b0b6-4622-ba4a-6b11f1747dde tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:34.597937 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-89e35c1a-0be6-461c-8eb2-c1e5e545914a req-4c4bfa55-b0b6-4622-ba4a-6b11f1747dde tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1677673 May 31 11:55:34.637135 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-89e35c1a-0be6-461c-8eb2-c1e5e545914a req-74f10a44-372d-4ea3-8984-77c5c7a76fd5 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0411880 May 31 11:55:38.023936 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9023a813-d5f4-494c-b81d-07b0d36fdb71 req-f362897c-3403-4f17-a137-ef83ea474b59 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:38.025353 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9023a813-d5f4-494c-b81d-07b0d36fdb71 req-f362897c-3403-4f17-a137-ef83ea474b59 service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1751757 May 31 11:55:38.072226 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-99846e5b-6525-40d8-ba66-038aa42c921f req-0150de37-e6fc-4d17-8ae4-084d0e5a044c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:38.073961 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-99846e5b-6525-40d8-ba66-038aa42c921f req-0150de37-e6fc-4d17-8ae4-084d0e5a044c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 944 time: 0.0889361 May 31 11:55:38.086659 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4bb97b2c-84e4-43e1-a3d1-d87ca9241440 req-5f8ab82f-32de-47f5-b79a-e5ada1712926 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:38.087672 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4bb97b2c-84e4-43e1-a3d1-d87ca9241440 req-5f8ab82f-32de-47f5-b79a-e5ada1712926 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0967832 May 31 11:55:38.138126 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-99846e5b-6525-40d8-ba66-038aa42c921f req-65ea6b86-253f-4e6d-80f8-198ab355f6ab tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0549202 May 31 11:55:38.183325 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4bb97b2c-84e4-43e1-a3d1-d87ca9241440 req-ab225df4-262f-4b77-928f-5d7261a83967 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0597255 May 31 11:55:38.197440 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-d0bb6e2b-9525-4a97-8878-f6eb3648a61f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:38.198881 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-d0bb6e2b-9525-4a97-8878-f6eb3648a61f service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 1097 time: 0.1103904 May 31 11:55:38.277218 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-7b6dd8a1-c668-4bce-9a93-635b577b7952 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 HTTP/1.1" status: 200 len: 217 time: 0.0679467 May 31 11:55:38.347471 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-4fd3f2bd-6cd0-49ba-8e55-becacaebd715 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:38.347784 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-4fd3f2bd-6cd0-49ba-8e55-becacaebd715 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 200 len: 855 time: 0.0636575 May 31 11:55:38.398808 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-cb7b711c-49a2-4964-9ab9-5c7b8cd6fe2f service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=709626b6-807b-4256-9031-1578948f1fe8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0441227 May 31 11:55:38.546600 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-a88f0bbd-e7db-47b2-93c7-2a5f5803e455 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1324413 May 31 11:55:38.686316 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ee3add7a-ab80-444f-8c0d-93bfb6221129 req-563aeb37-c817-44cd-8271-e6707f3fcc96 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1277611 May 31 11:55:39.011757 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:39.381445 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d93c842-9feb-4ce7-bcc2-110c923f9db7 req-6f7dea60-a732-4c55-ab56-6dcaa2a85992 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:39.383888 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8d93c842-9feb-4ce7-bcc2-110c923f9db7 req-6f7dea60-a732-4c55-ab56-6dcaa2a85992 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 944 time: 0.1012526 May 31 11:55:39.432841 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-123afd08-3c5f-41c8-9f0e-5c57163d3a9e req-6f193442-09cf-42f4-8a3b-1758051ef6db tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:39.434733 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-123afd08-3c5f-41c8-9f0e-5c57163d3a9e req-6f193442-09cf-42f4-8a3b-1758051ef6db tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1225922 May 31 11:55:39.445006 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8d93c842-9feb-4ce7-bcc2-110c923f9db7 req-49e23ef9-4a17-484e-b460-5568af27d876 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0553586 May 31 11:55:39.492308 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-123afd08-3c5f-41c8-9f0e-5c57163d3a9e req-ee3081f1-3bb3-416b-a874-cca8d14565ad tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0512393 May 31 11:55:40.126938 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-5be3cf79-af00-4977-ada9-46b1b22c1278 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0702617 May 31 11:55:40.288101 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-43dc2562-7621-48f0-82be-06b7a132a0c0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:40.289607 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-43dc2562-7621-48f0-82be-06b7a132a0c0 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 1098 time: 0.0756412 May 31 11:55:40.441497 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-94e81c3d-0f59-45eb-9fab-cf9a1072b9e2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:40.442947 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-94e81c3d-0f59-45eb-9fab-cf9a1072b9e2 service neutron] 10.222.0.47 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 916 time: 0.1328156 May 31 11:55:40.519321 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-d8499dc4-7547-42a6-897a-cc54a18c77c8 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=c4a233f2-9446-42a7-964d-1dc2496c938a HTTP/1.1" status: 200 len: 217 time: 0.0620315 May 31 11:55:40.594225 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-a72e90f5-d144-4e98-a12b-f802f7d05a34 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:40.595676 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-a72e90f5-d144-4e98-a12b-f802f7d05a34 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 200 len: 855 time: 0.0636182 May 31 11:55:40.647729 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-12f53b68-ce50-4173-b97c-7c5476937ee8 req-79e7b709-aa4e-44d5-99ba-e54c34399dbb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:40.649222 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-12f53b68-ce50-4173-b97c-7c5476937ee8 req-79e7b709-aa4e-44d5-99ba-e54c34399dbb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 944 time: 0.0920031 May 31 11:55:40.692492 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-c423a588-9202-4372-9345-3a7dd077f518 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=709626b6-807b-4256-9031-1578948f1fe8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0857697 May 31 11:55:40.709279 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-12f53b68-ce50-4173-b97c-7c5476937ee8 req-63a87a09-527d-432b-aa3c-b97a21b0e3ac tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0538335 May 31 11:55:40.935468 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5fe959e3-2edd-4e87-a118-309e5a513ac7 req-8d937702-f6cb-41d4-92f8-43458d50d6c3 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:40.936939 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5fe959e3-2edd-4e87-a118-309e5a513ac7 req-8d937702-f6cb-41d4-92f8-43458d50d6c3 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.3282616 May 31 11:55:40.940317 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-dcb46efe-13e4-4e65-b12f-7959514aeacf service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2331591 May 31 11:55:40.983072 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5fe959e3-2edd-4e87-a118-309e5a513ac7 req-38aec142-7779-4a0b-bc6b-ebafa43728ad tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0400636 May 31 11:55:43.070273 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:55:43.205399 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:43.219511 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8', 'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'port_id': '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8', 'mac_address': 'fa:16:3e:0b:78:b5', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 356, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c58bc8a-d411-461e-a483-39773ff0d6f4', 'ip_address': '172.20.0.14'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:55:43.310994 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:43.329903 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:55:43.347170 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:43.468843 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:43.471458 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:43.536974 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:43.546936 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.547275 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.547613 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.547945 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.548287 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.548597 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.548918 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.549254 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.549592 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.549934 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.550254 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.550569 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:55:43.555117 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:44.501105 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdc86bd1-d3f5-47d2-9ad3-2b579089e069 req-226c94b6-e48d-41c7-a74e-372749799404 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1831145 May 31 11:55:44.540264 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7418b4b-7d09-48ed-b7db-b16a9420517c req-7bbdf0cc-5227-4308-94e5-e0b1fbf79ed6 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:44.541760 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a7418b4b-7d09-48ed-b7db-b16a9420517c req-7bbdf0cc-5227-4308-94e5-e0b1fbf79ed6 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 945 time: 0.0869193 May 31 11:55:44.557758 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ccd03d14-9e83-4e1f-8e08-a7bcce023833 req-f9919881-bbc1-42a1-9a92-560ffa3043f2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:44.559532 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ccd03d14-9e83-4e1f-8e08-a7bcce023833 req-f9919881-bbc1-42a1-9a92-560ffa3043f2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0908854 May 31 11:55:44.591000 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a7418b4b-7d09-48ed-b7db-b16a9420517c req-0c523b4e-d810-4201-9425-f0843a39f21a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0433860 May 31 11:55:44.608129 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ccd03d14-9e83-4e1f-8e08-a7bcce023833 req-2c66c7a4-657d-424a-837c-828b588df447 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401855 May 31 11:55:44.756885 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['064a6ea1-ceec-462e-b73a-4e2c88f7a5b8'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:55:48.356855 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f8d41b5-0fc5-4dff-8c05-68e15e35d38b req-0b27ae0f-4ead-4178-ad4f-a8caabfc58c8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:48.358337 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1f8d41b5-0fc5-4dff-8c05-68e15e35d38b req-0b27ae0f-4ead-4178-ad4f-a8caabfc58c8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 945 time: 0.0911808 May 31 11:55:48.361276 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-700947be-6711-4892-aa6a-0a69d1a516d4 req-8c54306f-b75d-41d5-8171-6495d499dc32 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:48.362738 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-700947be-6711-4892-aa6a-0a69d1a516d4 req-8c54306f-b75d-41d5-8171-6495d499dc32 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0915811 May 31 11:55:48.403029 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1f8d41b5-0fc5-4dff-8c05-68e15e35d38b req-d700e1f8-f50f-4afe-a1c8-220a103f406e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0388854 May 31 11:55:48.408819 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-700947be-6711-4892-aa6a-0a69d1a516d4 req-19703c3c-f74a-4571-99f0-d2be6661fc0d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396304 May 31 11:55:50.919676 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-94281776-f037-4332-bb45-97de44a284bb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:51.952990 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63a1d261-9b24-48f7-b15a-18295b07b3ba req-23b7af3f-00a9-4051-b16d-e48cecf1d8a2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:51.954711 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-63a1d261-9b24-48f7-b15a-18295b07b3ba req-23b7af3f-00a9-4051-b16d-e48cecf1d8a2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 945 time: 0.0820649 May 31 11:55:51.959459 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc60949d-6710-428e-b8b3-ea03c8a417c6 req-43c7fd0a-0762-4841-85fc-c6c71a916a39 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:51.961005 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc60949d-6710-428e-b8b3-ea03c8a417c6 req-43c7fd0a-0762-4841-85fc-c6c71a916a39 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0838032 May 31 11:55:52.001519 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-63a1d261-9b24-48f7-b15a-18295b07b3ba req-3b0e7a20-dd16-4d65-9551-b4ad2a9c52be tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409458 May 31 11:55:52.004919 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc60949d-6710-428e-b8b3-ea03c8a417c6 req-e7571e61-4cca-42a2-8ef3-be22cdb05525 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0381510 May 31 11:55:52.298809 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:55:52.457233 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:55:52.462396 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:55:52.462777 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:52.581876 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:52.610457 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '85c447cc-29b9-4fd4-a3f6-d7219276b581', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:55:52.612685 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-d2d7ec19-c41a-4a8c-ae2a-4dfbc4453285" -d '{"events": [{"server_uuid": "85c447cc-29b9-4fd4-a3f6-d7219276b581", "name": "network-vif-plugged", "status": "completed", "tag": "064a6ea1-ceec-462e-b73a-4e2c88f7a5b8"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:55:52.621462 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:55:52.662143 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:55:52 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-fd769780-974b-4811-b698-5ff9714d1499 x-openstack-request-id: req-fd769780-974b-4811-b698-5ff9714d1499 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:55:52.664302 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "85c447cc-29b9-4fd4-a3f6-d7219276b581", "name": "network-vif-plugged", "status": "completed", "tag": "064a6ea1-ceec-462e-b73a-4e2c88f7a5b8", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:55:52.664742 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-fd769780-974b-4811-b698-5ff9714d1499 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:55:52.665333 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '85c447cc-29b9-4fd4-a3f6-d7219276b581', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8', 'code': 200} May 31 11:55:52.709011 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:52.711618 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:52.779714 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:52.782706 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:53.205353 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8416455e-b91d-4687-9a9c-79d4d3a01d9d req-9296b42a-a714-4133-a296-70dc6fa8d1b6 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:53.208028 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8416455e-b91d-4687-9a9c-79d4d3a01d9d req-9296b42a-a714-4133-a296-70dc6fa8d1b6 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0851004 May 31 11:55:53.224580 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eeae8b46-6238-4f4a-9eec-ff06a6cb98b4 req-2dadc37b-dd18-4317-8628-b7ab67302b8e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:53.226167 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-eeae8b46-6238-4f4a-9eec-ff06a6cb98b4 req-2dadc37b-dd18-4317-8628-b7ab67302b8e service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 1099 time: 0.1556976 May 31 11:55:53.233693 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3f5b57e0-28f0-4eb1-bc15-5b94cf7db635 req-6a566dfd-dbbc-41db-a43f-8cec632ed741 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:53.235091 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3f5b57e0-28f0-4eb1-bc15-5b94cf7db635 req-6a566dfd-dbbc-41db-a43f-8cec632ed741 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 946 time: 0.1111510 May 31 11:55:53.268450 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8416455e-b91d-4687-9a9c-79d4d3a01d9d req-57a2ddbe-5718-4ad9-ab12-d2972a99f58c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0515664 May 31 11:55:53.308131 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-eeae8b46-6238-4f4a-9eec-ff06a6cb98b4 req-190740c4-7823-4204-b5f3-2d3851fda881 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0715919 May 31 11:55:53.317876 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3f5b57e0-28f0-4eb1-bc15-5b94cf7db635 req-c7749f20-b23b-4985-80ad-5a3d9adf94ea tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0657222 May 31 11:55:53.354300 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:53.475836 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:55:53.479924 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-4b8893df-4a4e-49e1-9f1b-f2b43360150b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:54.482061 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-427aa9f7-2f6f-4695-b41f-8dd1f198df1f req-50c24767-0586-4775-bb90-5ad01a76bbe8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:54.483523 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-427aa9f7-2f6f-4695-b41f-8dd1f198df1f req-50c24767-0586-4775-bb90-5ad01a76bbe8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1065915 May 31 11:55:54.519351 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5c251cd2-e202-4332-8a46-d9d83d76dc92 req-7601e7b3-cd95-4bcb-9718-1ddbfaa63133 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:54.527451 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5c251cd2-e202-4332-8a46-d9d83d76dc92 req-7601e7b3-cd95-4bcb-9718-1ddbfaa63133 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 946 time: 0.0946696 May 31 11:55:54.576327 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-427aa9f7-2f6f-4695-b41f-8dd1f198df1f req-0ebae723-39be-4d6e-af5a-ef1744ec5db8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0869296 May 31 11:55:54.598514 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5c251cd2-e202-4332-8a46-d9d83d76dc92 req-06294536-72fd-42d2-ad70-634828364608 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0643852 May 31 11:55:54.620716 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:55.255668 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59556) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:55.372599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:55.386926 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:55.390858 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Usage tracker for resource:security_group_rule and tenant:2d9db0340dde4fa0a24ea9663ff1510e is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:55.393713 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:55.400525 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:55.485366 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:55.548303 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:55.557854 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:55.596163 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=2d9db0340dde4fa0a24ea9663ff1510e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3394170 May 31 11:55:55.597888 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=789616e1-fa56-47d0-bd21-70880f1d8c58,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:55.599066 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 59576) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:55.602039 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:55.686266 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c5dcfcd8-1fcd-4158-8648-ddac36f3a2c4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:55.759426 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c5dcfcd8-1fcd-4158-8648-ddac36f3a2c4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:55.798462 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad9dbe77-d11e-4b8c-bc5e-a85b9307f6f3 req-e443aac3-0c46-4de5-8c17-93fb60c3bd1f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:55.799924 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ad9dbe77-d11e-4b8c-bc5e-a85b9307f6f3 req-e443aac3-0c46-4de5-8c17-93fb60c3bd1f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 946 time: 0.0814853 May 31 11:55:55.826176 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-917b5cf0-8d55-4660-bfcf-de71917397f9 req-d8a35d08-5340-40bd-a274-1283c15ea615 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:55.827654 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-917b5cf0-8d55-4660-bfcf-de71917397f9 req-d8a35d08-5340-40bd-a274-1283c15ea615 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1288300 May 31 11:55:55.848504 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ad9dbe77-d11e-4b8c-bc5e-a85b9307f6f3 req-8862f3a7-9083-4520-9ae3-2e17a094c0ee tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0413885 May 31 11:55:55.886138 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-917b5cf0-8d55-4660-bfcf-de71917397f9 req-84e1cd96-455f-49c0-8cdf-bb1955d4ab82 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0511358 May 31 11:55:55.920001 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c5dcfcd8-1fcd-4158-8648-ddac36f3a2c4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:55.925331 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:55.940572 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c5dcfcd8-1fcd-4158-8648-ddac36f3a2c4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=789616e1-fa56-47d0-bd21-70880f1d8c58,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:55.943750 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c5dcfcd8-1fcd-4158-8648-ddac36f3a2c4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:55.952537 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c5dcfcd8-1fcd-4158-8648-ddac36f3a2c4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:55.953784 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c5dcfcd8-1fcd-4158-8648-ddac36f3a2c4 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/789616e1-fa56-47d0-bd21-70880f1d8c58 HTTP/1.1" status: 204 len: 168 time: 0.3529108 May 31 11:55:56.079574 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.134856 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 59638) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:57.239691 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-404047115-network', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:55:57.244420 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:57.248168 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Usage tracker for resource:network and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:57.251066 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:57.257655 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:57.330991 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Made reservation on behalf of 39ae37364a184ae7ae341ea7b2847c35 for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:55:57.333288 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.346066 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.362930 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:57.366716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Usage tracker for resource:security_group_rule and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:57.369520 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:57.375230 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:57.462276 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.537234 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.541803 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0d6a6688-f7d4-4430-a59a-4f3d41628ccb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:57.564976 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] vlan segment allocate from pool success with {'vlan_id': 366, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:55:57.577760 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7392633c-3cf7-4366-bb3e-fd78503c0a30,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:57.581725 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:57.582635 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.582962 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Added segment 061f4d79-99bf-47ba-a95e-2ecb79298c69 of type vlan for network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb May 31 11:55:57.609851 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.731146 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.732490 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:57.733065 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:57.738002 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:57.836410 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Pushing event updated for resources: {'Network': ['ID=0fae71a7-8e60-4f7d-a467-822e3d8d87fb,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:57.839580 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:57.846568 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:55:57.912172 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.7734642 May 31 11:55:57.913210 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59648) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:57.919797 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Request body: {'subnet': {'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-404047115-subnet', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:55:57.924399 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:57.929218 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Usage tracker for resource:subnet and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:57.932330 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:57.938116 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:58.005721 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Made reservation on behalf of 39ae37364a184ae7ae341ea7b2847c35 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:55:58.008062 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:58.463657 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:58.468159 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-44d696c5-1287-4eac-a8e6-60ff684f0b79 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:58.546623 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Pushing event updated for resources: {'Subnet': ['ID=672ec900-b55c-44a0-9ed9-7e27beeace12,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:58.550569 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:58.555781 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Checking segment: {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:55:58.556159 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Network hostable DHCP agents. Network: 0fae71a7-8e60-4f7d-a467-822e3d8d87fb, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:55:58.650407 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4aa903b6-bdf6-4c61-bbd7-bd2a5eb6dd7d req-96bf4d40-1a30-4f08-ac04-35dec6947fbd tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:58.651855 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4aa903b6-bdf6-4c61-bbd7-bd2a5eb6dd7d req-96bf4d40-1a30-4f08-ac04-35dec6947fbd tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0805261 May 31 11:55:58.652872 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:55:58.698684 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4aa903b6-bdf6-4c61-bbd7-bd2a5eb6dd7d req-df539a87-fadc-4000-9601-cf5184fca27a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0398533 May 31 11:55:58.725342 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 0fae71a7-8e60-4f7d-a467-822e3d8d87fb) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:55:58.735187 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-68e3ad63-4b4a-4309-91ab-ccb45ba51a64 None None] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:55:58.736029 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:58.736729 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:58.851797 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 850 time: 0.9365571 May 31 11:55:58.854326 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59660) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:55:58.859297 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:55:58.863787 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:55:58.868518 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Usage tracker for resource:router and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:55:58.871382 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:55:58.877066 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:55:58.959097 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 11:55:58.959947 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:58.971229 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:58.972443 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:58.989483 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:59.118985 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Allocated IP 172.20.0.2 (0fae71a7-8e60-4f7d-a467-822e3d8d87fb/672ec900-b55c-44a0-9ed9-7e27beeace12/447dd278-807b-43af-8f79-ec082f9f9049) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:55:59.336921 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:59.347344 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Transition to ACTIVE for port object 447dd278-807b-43af-8f79-ec082f9f9049 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:55:59.368355 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Transition to ACTIVE for port object 447dd278-807b-43af-8f79-ec082f9f9049 will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:55:59.436894 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:59.475333 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:59.486912 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:55:59.555251 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Pushing event updated for resources: {'Port': ['ID=447dd278-807b-43af-8f79-ec082f9f9049,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:55:59.567397 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:55:59.577918 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:55:59.579264 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:55:59.583542 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:55:59.627325 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62ae8c3d-7e40-4852-b236-938bacb0a37b req-b7ccad0f-8306-4ac6-ac61-73496a736ccc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:55:59.628866 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-62ae8c3d-7e40-4852-b236-938bacb0a37b req-b7ccad0f-8306-4ac6-ac61-73496a736ccc service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 1098 time: 0.1713135 May 31 11:55:59.697756 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-62ae8c3d-7e40-4852-b236-938bacb0a37b req-fc1a7c2b-da10-41ca-a4b9-a2105d2770e8 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0561180 May 31 11:55:59.721014 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Attempting to bind port 447dd278-807b-43af-8f79-ec082f9f9049 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:55:59.721385 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Attempting to bind port 447dd278-807b-43af-8f79-ec082f9f9049 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:55:59.721773 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Attempting to bind port 447dd278-807b-43af-8f79-ec082f9f9049 on network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:55:59.728354 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 55, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:55:59.728747 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Checking segment: {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:55:59.729133 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:59.729536 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Bound using segment: {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:55:59.730033 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Bound port: 447dd278-807b-43af-8f79-ec082f9f9049, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:55:59.730547 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:59.834570 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] For port 447dd278-807b-43af-8f79-ec082f9f9049, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:55:59.839383 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] For port 447dd278-807b-43af-8f79-ec082f9f9049, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=447dd278-807b-43af-8f79-ec082f9f9049,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:55:59.935460 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:55:59.970342 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:00.014814 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4660967-96ce-4829-b183-b411e9fc4eb2 req-f1100e1a-f487-4128-a59d-f0faccf355c4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:00.016143 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a4660967-96ce-4829-b183-b411e9fc4eb2 req-f1100e1a-f487-4128-a59d-f0faccf355c4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1904175 May 31 11:56:00.046188 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:00.052389 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:00.119920 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a4660967-96ce-4829-b183-b411e9fc4eb2 req-5fa86fc0-8b31-4e8f-a784-f0d666ea7df9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0916843 May 31 11:56:00.153852 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Pushing event updated for resources: {'Port': ['ID=447dd278-807b-43af-8f79-ec082f9f9049,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:00.157498 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:00.179314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Allocated IP 172.27.1.214 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/1b4b9cf5-650e-45b0-b762-4b42ac55ad1d) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:56:00.386930 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:00.563656 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:00.567527 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-21893033-72d8-447d-9cc6-98037dc755a4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:00.637406 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 447dd278-807b-43af-8f79-ec082f9f9049 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:00.644071 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:00.648313 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Pushing event updated for resources: {'Port': ['ID=1b4b9cf5-650e-45b0-b762-4b42ac55ad1d,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:00.656036 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:00.657955 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:00.658621 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:56:00.695877 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c238b699-e488-49b4-b530-9c34724d8753 req-dbcdf8aa-5dd0-4b45-9bce-90725ced3fc7 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:00.703769 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-dbcdf8aa-5dd0-4b45-9bce-90725ced3fc7 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 815 time: 0.2142997 May 31 11:56:00.718193 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-ac39b217-f37c-49da-8a14-ab7b3f67b6f2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/quotas/7dea63b6c4a340d7b361fe91ca6b8281 HTTP/1.1" status: 200 len: 375 time: 0.0131922 May 31 11:56:00.811254 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-45d89b50-b736-45a6-9dcb-89f2095bc313 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0905337 May 31 11:56:00.828868 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 447dd278-807b-43af-8f79-ec082f9f9049 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:01.051600 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-ad023b12-82bc-4ff9-ae4e-302871b0e2fe service neutron] 10.222.0.48 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2325404 May 31 11:56:01.230139 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-68b09f02-3af8-4a00-8c2e-e0fbd99301d5 service neutron] 10.222.0.48 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1715093 May 31 11:56:01.295577 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4501a0c7-0ce8-4458-94c5-03970ec69b35 None None] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:56:01.366204 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:01.510134 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] Provisioning for port 447dd278-807b-43af-8f79-ec082f9f9049 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:01.528021 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] Provisioning complete for port 447dd278-807b-43af-8f79-ec082f9f9049 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:01.528643 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:01.528804 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1161c553-5155-491c-ac17-0df3a8d2d9ed req-bac0d1f0-bee4-4680-809a-03507d6cba51 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:01.530334 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1161c553-5155-491c-ac17-0df3a8d2d9ed req-bac0d1f0-bee4-4680-809a-03507d6cba51 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.2551241 May 31 11:56:01.582221 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-733cb69e-9416-421f-a3a3-504e1c82b8c0 req-5236aa07-18e4-4afd-a237-83dc6761bffe tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 210 time: 0.1222544 May 31 11:56:01.710164 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1161c553-5155-491c-ac17-0df3a8d2d9ed req-eeb47905-70d8-4e20-8a2d-3b7c94c80be4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1719656 May 31 11:56:01.755474 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:01.797797 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] For port 447dd278-807b-43af-8f79-ec082f9f9049, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=447dd278-807b-43af-8f79-ec082f9f9049,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:01.874650 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:01.897465 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:01.899632 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:01.989825 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19c5308d-1a1f-47c8-a8db-55871401d686 req-b5208835-a2a1-4ebe-8175-92c279693632 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:02.006755 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-19c5308d-1a1f-47c8-a8db-55871401d686 req-b5208835-a2a1-4ebe-8175-92c279693632 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1623204 May 31 11:56:02.072974 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-19c5308d-1a1f-47c8-a8db-55871401d686 req-83e65229-25ee-4a9c-9d49-7dc3e462c2aa tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0642366 May 31 11:56:02.136839 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] Pushing event updated for resources: {'Port': ['ID=447dd278-807b-43af-8f79-ec082f9f9049,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:02.149676 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.250s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:02.173258 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-33499f0b-3b31-4d34-b4a9-cbd38fbc119f None None] Provisioning complete for port 447dd278-807b-43af-8f79-ec082f9f9049 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:02.175842 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-33499f0b-3b31-4d34-b4a9-cbd38fbc119f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:02.319935 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Router d195a538-22a1-483f-a3e5-9007c6338ece is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99039) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 11:56:02.440303 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 447dd278-807b-43af-8f79-ec082f9f9049 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:02.508207 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:56:02.511925 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 447dd278-807b-43af-8f79-ec082f9f9049 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:02.512560 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:02.517012 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Sync routers for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:56:02.533934 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:56:02.543088 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:55:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:02.640045 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:02.641252 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 926 time: 3.7858672 May 31 11:56:02.643918 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59762) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:02.739351 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:56:02.752404 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ed0c960b-cd75-4bf5-94d9-3d920f73d970 req-5820b375-a879-44c6-b8e1-e9e51ac6d297 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 210 time: 0.0755539 May 31 11:56:03.059500 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:01Z', 'revision_number': 3, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:00Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:03.063407 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Checking router: d195a538-22a1-483f-a3e5-9007c6338ece for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:56:03.063760 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Updating router d195a538-22a1-483f-a3e5-9007c6338ece port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:56:03.119869 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:03.549064 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:03.560658 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Transition to ACTIVE for port object 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:03.568535 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:03.674806 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:03.689293 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-297bb2d7-0ad5-499d-86e0-0b702be878b3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:03.755826 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Pushing event updated for resources: {'Port': ['ID=1b4b9cf5-650e-45b0-b762-4b42ac55ad1d,revision_number=2']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:03.814585 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.815774 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.816260 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.816736 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.817098 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Allocated IP 172.20.0.1 (0fae71a7-8e60-4f7d-a467-822e3d8d87fb/672ec900-b55c-44a0-9ed9-7e27beeace12/6b60438b-78d8-4039-a0ce-c42d04559f08) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:56:03.818939 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.818939 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.818939 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.818939 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.818939 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.820336 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.820809 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.821269 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:03.828688 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Attempting to bind port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:56:03.829447 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Attempting to bind port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:56:03.829986 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Attempting to bind port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:03.836847 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:03.837623 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 55, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:56:03.837994 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:56:03.838558 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:03.839107 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:56:03.839733 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Bound port: 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:56:03.840275 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:03.931850 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] For port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:56:03.935388 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] For port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=1b4b9cf5-650e-45b0-b762-4b42ac55ad1d,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:56:03.982868 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4287501d-1594-4bdf-94b6-c2bd2441a616 req-f76bd0cf-4de0-4299-8ac7-34b7532d1665 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 210 time: 0.0559123 May 31 11:56:04.042691 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.046204 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.052172 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:04.059328 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Transition to ACTIVE for port object 6b60438b-78d8-4039-a0ce-c42d04559f08 will not be triggered until provisioned by entity DHCP. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:04.139748 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:56:04.140411 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:55:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:04.165995 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.170198 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:04.246609 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Pushing event updated for resources: {'Port': ['ID=1b4b9cf5-650e-45b0-b762-4b42ac55ad1d,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:04.251156 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-b769863e-a46f-4f30-882a-9b1d37ce65ad None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.199s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:04.263506 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:04.267014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:04.271459 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:56:04.322156 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Pushing event updated for resources: {'Port': ['ID=6b60438b-78d8-4039-a0ce-c42d04559f08,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:04.340139 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:04.344523 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.457455 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:56:04.543104 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.611359 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-07a38022-ce14-4680-8bd5-a88fa7f1ae0b None None] Provisioning for port 6b60438b-78d8-4039-a0ce-c42d04559f08 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:04.616084 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-07a38022-ce14-4680-8bd5-a88fa7f1ae0b None None] Provisioning complete for port 6b60438b-78d8-4039-a0ce-c42d04559f08 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:04.616602 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-07a38022-ce14-4680-8bd5-a88fa7f1ae0b None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.670868 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-07a38022-ce14-4680-8bd5-a88fa7f1ae0b None None] Port 6b60438b-78d8-4039-a0ce-c42d04559f08 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:56:04.756885 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:01Z', 'revision_number': 3, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:03Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:04.761925 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-19f692ab-8710-4d9a-b05a-b76ee9ad3bdd None None] The sync data for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi is: [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:01Z', 'revision_number': 3, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:03Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:56:04.767415 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.847809 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.873994 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d1e43bb2-bc71-4125-a4cf-b90ad01cbedb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:04.957394 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Pushing event updated for resources: {'Port': ['ID=6b60438b-78d8-4039-a0ce-c42d04559f08,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:04.960280 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:04.965334 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:05.203768 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:05.204928 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:05.415306 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Router d195a538-22a1-483f-a3e5-9007c6338ece has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:56:05.500514 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-b5937b4b-d363-4f3b-9e2b-eda7b2431db8 None None] Provisioning complete for port 6b60438b-78d8-4039-a0ce-c42d04559f08 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:05.501043 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b5937b4b-d363-4f3b-9e2b-eda7b2431db8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:05.562433 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-b5937b4b-d363-4f3b-9e2b-eda7b2431db8 None None] Port 6b60438b-78d8-4039-a0ce-c42d04559f08 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:56:05.607371 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:56:05.616199 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] 10.222.0.48 "PUT /v2.0/routers/d195a538-22a1-483f-a3e5-9007c6338ece/add_router_interface HTTP/1.1" status: 200 len: 503 time: 2.9713507 May 31 11:56:06.305891 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Sync routers for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:56:06.318989 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:56:06.318989 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:55:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:06.472198 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:06.564205 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:56:06.641705 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59840) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:06.648273 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-774559787-network', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:56:06.653719 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:06.656875 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Usage tracker for resource:network and tenant:c8035d3733414b75bde96faf369f72d2 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:06.659941 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:06.666658 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:06.726937 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:06.748399 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Made reservation on behalf of c8035d3733414b75bde96faf369f72d2 for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:56:06.751202 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:06.752775 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:06.753155 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:06.768829 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:06.785055 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:06.790054 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Usage tracker for resource:security_group_rule and tenant:c8035d3733414b75bde96faf369f72d2 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:06.792824 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:06.798947 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:06.877960 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f65fbfc6-2a16-4c57-bb8a-086cfcec484e req-9b41dc41-7fe9-42e0-a1ef-12fa0ea438c0 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 210 time: 0.0432198 May 31 11:56:06.980182 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:06.999016 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.027149 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=1b4b9cf5-650e-45b0-b762-4b42ac55ad1d,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:07.048275 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:03Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 2, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:07.048275 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Checking router: d195a538-22a1-483f-a3e5-9007c6338ece for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:56:07.048275 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Updating router d195a538-22a1-483f-a3e5-9007c6338ece port 6b60438b-78d8-4039-a0ce-c42d04559f08 binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:56:07.068682 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.074470 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:07.102318 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.102834 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] vlan segment allocate from pool success with {'vlan_id': 376, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:56:07.111811 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=78fe6823-3c2b-4873-b2f4-33f35793a5dd,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:07.115422 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:07.118645 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.122868 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.122868 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Added segment 0eba2510-55a0-4815-bcb1-2dc142aabf11 of type vlan for network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f May 31 11:56:07.123364 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-d3f4cc04-0212-4978-97d8-90420cca8650 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:07.151734 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.274721 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=1b4b9cf5-650e-45b0-b762-4b42ac55ad1d,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:07.277808 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:07.297297 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.298739 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:07.299241 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.304269 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5e6624d9-4791-4adb-b7b4-e0ee81f022f0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:07.433343 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Pushing event updated for resources: {'Network': ['ID=87442cb7-bfc4-4bf2-9d82-09a2c5a5344f,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:07.483438 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:07.510872 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:56:07.560730 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c238b699-e488-49b4-b530-9c34724d8753 req-5f1312f3-50b4-4eee-87e6-46320ff6ec00 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:07.562157 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-5f1312f3-50b4-4eee-87e6-46320ff6ec00 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 815 time: 0.1668355 May 31 11:56:07.564253 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.9213898 May 31 11:56:07.566879 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59850) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:07.572856 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Request body: {'subnet': {'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-774559787-subnet', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:56:07.582103 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Request body: {'port': {'device_id': '231d5bcd-ca61-435d-8029-4d0537a194ad', 'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'admin_state_up': True, 'tenant_id': '7dea63b6c4a340d7b361fe91ca6b8281'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:56:07.605714 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:07.611715 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.615156 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Usage tracker for resource:subnet and tenant:c8035d3733414b75bde96faf369f72d2 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:07.635900 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Transition to ACTIVE for port object 6b60438b-78d8-4039-a0ce-c42d04559f08 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:07.670121 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:07.682528 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:07.755172 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.758920 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:07.760441 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:07.768919 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:07.811823 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Made reservation on behalf of c8035d3733414b75bde96faf369f72d2 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:56:07.814225 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.900056 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Made reservation on behalf of 7dea63b6c4a340d7b361fe91ca6b8281 for: {'port': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:56:07.910619 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:07.941195 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5821890d-78aa-444f-b8f1-404627382df9 None None] Pushing event updated for resources: {'Port': ['ID=6b60438b-78d8-4039-a0ce-c42d04559f08,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:07.979849 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5821890d-78aa-444f-b8f1-404627382df9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.219s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:08.035858 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Attempting to bind port 6b60438b-78d8-4039-a0ce-c42d04559f08 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:56:08.036502 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Attempting to bind port 6b60438b-78d8-4039-a0ce-c42d04559f08 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:56:08.037049 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Attempting to bind port 6b60438b-78d8-4039-a0ce-c42d04559f08 on network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:08.044578 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 55, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:56:08.045235 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Checking segment: {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:56:08.045882 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:08.046474 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Bound using segment: {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:56:08.047133 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Bound port: 6b60438b-78d8-4039-a0ce-c42d04559f08, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:56:08.049243 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:08.122325 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-47b5ea47-a5ff-4a5a-8b95-a75cb4521521 req-98d85122-c3bb-4fdf-a2e4-d3717ab9b9ad tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 210 time: 0.0450299 May 31 11:56:08.158711 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] For port 6b60438b-78d8-4039-a0ce-c42d04559f08, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:56:08.162241 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] For port 6b60438b-78d8-4039-a0ce-c42d04559f08, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=6b60438b-78d8-4039-a0ce-c42d04559f08,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:56:08.218608 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Allocated IP 172.20.0.10 (709626b6-807b-4256-9031-1578948f1fe8/4c58bc8a-d411-461e-a483-39773ff0d6f4/150f3a3b-28b5-435a-b826-0eee61f6b143) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:56:08.252766 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:08.257835 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:08.355366 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:56:08.355995 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:55:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:08.432337 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5821890d-78aa-444f-b8f1-404627382df9 None None] Pushing event updated for resources: {'Port': ['ID=6b60438b-78d8-4039-a0ce-c42d04559f08,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:08.449843 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5821890d-78aa-444f-b8f1-404627382df9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.192s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:08.493711 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:08.735508 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:08.742694 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:56:08.795783 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:08.795783 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:08.867765 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:08.873752 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c7078480-2e01-4340-8ae0-a018c426a367 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:08.945684 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:09.050901 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Pushing event updated for resources: {'Subnet': ['ID=c01a063a-2236-4017-91be-4828f1866f28,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:09.056763 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-831261ff-fdca-4823-a636-b82272c87133 req-41d506da-a5ed-4ac3-a259-3b82e4fb9c2f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:09.057190 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['c4a233f2-9446-42a7-964d-1dc2496c938a', '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:56:09.058904 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.185s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:09.060733 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.102s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:09.062642 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-831261ff-fdca-4823-a636-b82272c87133 req-41d506da-a5ed-4ac3-a259-3b82e4fb9c2f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1707513 May 31 11:56:09.107993 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Network 709626b6-807b-4256-9031-1578948f1fe8 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:09.115198 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:09.118785 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:56:09.123944 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-1e94677f-644f-497b-ab2b-346888df556f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:09.125402 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-1e94677f-644f-497b-ab2b-346888df556f service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1099 time: 0.1673670 May 31 11:56:09.129484 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:09.136149 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-831261ff-fdca-4823-a636-b82272c87133 req-975e582e-de9b-4d87-8fcd-49357525da8d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0726125 May 31 11:56:09.197849 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Checking segment: {'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:56:09.197849 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Network hostable DHCP agents. Network: 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:56:09.242906 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Pushing event updated for resources: {'Port': ['ID=150f3a3b-28b5-435a-b826-0eee61f6b143,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:09.252024 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.191s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:09.260617 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:09.261460 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:09.389974 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-0c230bc6-8558-4a28-be10-153527f11b60 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:09.391406 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-0c230bc6-8558-4a28-be10-153527f11b60 service neutron] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 920 time: 0.2543528 May 31 11:56:09.400162 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:56:09.435700 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.8583355 May 31 11:56:09.465120 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Request body: {'port': {'device_id': '231d5bcd-ca61-435d-8029-4d0537a194ad', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:56:09.465341 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-6857b2ce-68e0-4197-969b-e9fccfa3d8b3 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778 HTTP/1.1" status: 200 len: 217 time: 0.0642023 May 31 11:56:09.482003 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4cf75d0c-4918-4ccb-814a-44d3183c8dbe req-1ce485e0-4c45-4582-bafc-0551438b21a1 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:09.488456 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cf75d0c-4918-4ccb-814a-44d3183c8dbe req-1ce485e0-4c45-4582-bafc-0551438b21a1 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 932 time: 0.1741781 May 31 11:56:09.549581 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:09.565264 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6b1b6f0d-7db9-4793-8335-4272edca690e None None] Network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:56:09.598872 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4cf75d0c-4918-4ccb-814a-44d3183c8dbe req-55904cc4-8b3c-4e1f-b51f-7102ba321e98 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1095638 May 31 11:56:09.603422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-9ba97bec-a65e-4d0f-83b6-ea751137c792 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:09.605006 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-9ba97bec-a65e-4d0f-83b6-ea751137c792 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.1140008 May 31 11:56:09.609594 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:08Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:09.614527 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:09.615145 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:09.635868 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:09.663846 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-837a0810-d631-485c-b692-d1f306d3b246 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0451617 May 31 11:56:09.678643 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a6dd34d2-7dbc-4331-84d1-f3d5d0b2295d None None] The sync data for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi is: [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:08Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:56:09.816055 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 850 time: 2.2480974 May 31 11:56:09.818525 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 59910) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:09.828819 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:56:09.848623 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:09.866902 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Usage tracker for resource:router and tenant:c8035d3733414b75bde96faf369f72d2 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:09.872613 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:09.881339 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-30181cd0-7cf8-49e2-8a51-7b7cce9f3e38 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2075653 May 31 11:56:09.883260 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:10.045843 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.046951 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.087602 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-273e9bda-15a7-4224-b106-51962fcaf5d7 req-b658765f-d8e8-48d7-a3e3-fae214c90663 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1966953 May 31 11:56:10.091100 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.114052 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-87442cb7-bfc4-4bf2-9d82-09a2c5a5344f', 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'fixed_ips': [{'subnet_id': 'c01a063a-2236-4017-91be-4828f1866f28'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 11:56:10.115941 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.168802 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 150f3a3b-28b5-435a-b826-0eee61f6b143 {{(pid=99039) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:56:10.176859 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.193130 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Transition to ACTIVE for port object 150f3a3b-28b5-435a-b826-0eee61f6b143 will not be triggered until provisioned by entity L2. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:10.281272 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.286836 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1fd57e48-14c6-4258-bd3b-08d74838d57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:10.365496 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Allocated IP 172.20.0.2 (87442cb7-bfc4-4bf2-9d82-09a2c5a5344f/c01a063a-2236-4017-91be-4828f1866f28/c495e266-af80-4662-84fa-21d967aef128) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:56:10.476305 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Pushing event updated for resources: {'Port': ['ID=150f3a3b-28b5-435a-b826-0eee61f6b143,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:10.539558 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.253s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:10.660347 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Attempting to bind port 150f3a3b-28b5-435a-b826-0eee61f6b143 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:56:10.660684 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Attempting to bind port 150f3a3b-28b5-435a-b826-0eee61f6b143 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:56:10.661020 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Attempting to bind port 150f3a3b-28b5-435a-b826-0eee61f6b143 on network 709626b6-807b-4256-9031-1578948f1fe8 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:10.663089 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 6b60438b-78d8-4039-a0ce-c42d04559f08 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:10.667196 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Port 150f3a3b-28b5-435a-b826-0eee61f6b143 on network 709626b6-807b-4256-9031-1578948f1fe8 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:56:10.667796 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Attempting to bind port 150f3a3b-28b5-435a-b826-0eee61f6b143 on network 709626b6-807b-4256-9031-1578948f1fe8 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:10.674974 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 56, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:56:10.675831 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Checking segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:56:10.676493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Bound using segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:56:10.677175 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Bound port: 150f3a3b-28b5-435a-b826-0eee61f6b143, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'}}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:56:10.677819 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.734373 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.746209 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Transition to ACTIVE for port object c495e266-af80-4662-84fa-21d967aef128 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:10.771180 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Transition to ACTIVE for port object c495e266-af80-4662-84fa-21d967aef128 will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:10.798890 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] For port 150f3a3b-28b5-435a-b826-0eee61f6b143, host hv-cinder-79123, cleared binding levels {{(pid=99039) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:56:10.802157 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] For port 150f3a3b-28b5-435a-b826-0eee61f6b143, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=150f3a3b-28b5-435a-b826-0eee61f6b143,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99039) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:56:10.825029 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.857440 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ac50e85-bced-4611-9137-6327a53fff29 req-cddd3b0a-7b44-46ab-8631-5c311d9b2d10 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:10.858823 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ac50e85-bced-4611-9137-6327a53fff29 req-cddd3b0a-7b44-46ab-8631-5c311d9b2d10 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.3604646 May 31 11:56:10.863687 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.869183 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:10.887769 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.893783 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-caa0b1ff-aba8-4648-9037-65a584c64e9e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:10.905556 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 6b60438b-78d8-4039-a0ce-c42d04559f08 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:10.912221 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ac50e85-bced-4611-9137-6327a53fff29 req-14f4d649-70fb-43a7-a484-caa3a1b92028 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0469239 May 31 11:56:10.932790 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 6b60438b-78d8-4039-a0ce-c42d04559f08 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:10.933335 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.994760 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:10.998055 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:10.998055 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] 10.222.0.47 "PUT /v2.0/ports/150f3a3b-28b5-435a-b826-0eee61f6b143 HTTP/1.1" status: 200 len: 1094 time: 1.5474796 May 31 11:56:11.002431 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '231d5bcd-ca61-435d-8029-4d0537a194ad', 'tag': '150f3a3b-28b5-435a-b826-0eee61f6b143'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:56:11.006554 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e" -d '{"events": [{"name": "network-changed", "server_uuid": "231d5bcd-ca61-435d-8029-4d0537a194ad", "tag": "150f3a3b-28b5-435a-b826-0eee61f6b143"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:56:11.038930 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Pushing event updated for resources: {'Port': ['ID=150f3a3b-28b5-435a-b826-0eee61f6b143,revision_number=3']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:11.043568 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.149s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:11.130854 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:11.145541 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:11.148338 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:56:11.156569 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Pushing event updated for resources: {'Port': ['ID=c495e266-af80-4662-84fa-21d967aef128,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:11.159294 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.290s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:11.308534 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Attempting to bind port c495e266-af80-4662-84fa-21d967aef128 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:56:11.308534 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Attempting to bind port c495e266-af80-4662-84fa-21d967aef128 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:56:11.308534 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Attempting to bind port c495e266-af80-4662-84fa-21d967aef128 on network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:11.323062 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 55, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:56:11.323433 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Checking segment: {'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:56:11.323838 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.324236 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Bound using segment: {'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:56:11.324708 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Bound port: c495e266-af80-4662-84fa-21d967aef128, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:56:11.325077 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.328661 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.359611 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.369859 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.438311 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 6b60438b-78d8-4039-a0ce-c42d04559f08, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=6b60438b-78d8-4039-a0ce-c42d04559f08,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:11.443806 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] For port c495e266-af80-4662-84fa-21d967aef128, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:56:11.451332 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] For port c495e266-af80-4662-84fa-21d967aef128, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=c495e266-af80-4662-84fa-21d967aef128,segment=NetworkSegment(0eba2510-55a0-4815-bcb1-2dc142aabf11),segment_id=0eba2510-55a0-4815-bcb1-2dc142aabf11)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:56:11.506328 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Allocated IP 172.27.1.203 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/4ccc4832-093a-44f0-a0bb-60f43b576309) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:56:11.554338 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.557254 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:11.594718 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.647178 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=6b60438b-78d8-4039-a0ce-c42d04559f08,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:11.650511 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:11.651191 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.046s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:11.681489 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.733910 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Pushing event updated for resources: {'Port': ['ID=c495e266-af80-4662-84fa-21d967aef128,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:11.737086 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:11.783051 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:11.787192 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:11.854698 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:11.857491 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:11.860814 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:56:11.920343 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Pushing event updated for resources: {'Port': ['ID=4ccc4832-093a-44f0-a0bb-60f43b576309,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:11.931100 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:12.320728 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:12.583397 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 6b60438b-78d8-4039-a0ce-c42d04559f08 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:12.632889 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-bd6beef8-9d05-42c9-9837-da7553dd9e11 None None] Network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:56:12.778762 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 6b60438b-78d8-4039-a0ce-c42d04559f08 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:12.779076 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:12.786994 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:12.810686 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-4b183515-e87b-4d75-9ea6-9faca4921938 None None] Provisioning for port c495e266-af80-4662-84fa-21d967aef128 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:13.019591 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device c495e266-af80-4662-84fa-21d967aef128 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:13.148793 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:56:11 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-46b67298-b5e4-469b-a057-bc69b17b4609 x-openstack-request-id: req-46b67298-b5e4-469b-a057-bc69b17b4609 {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:56:13.149850 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "231d5bcd-ca61-435d-8029-4d0537a194ad", "tag": "150f3a3b-28b5-435a-b826-0eee61f6b143", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:56:13.150361 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-46b67298-b5e4-469b-a057-bc69b17b4609 {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:56:13.151274 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '231d5bcd-ca61-435d-8029-4d0537a194ad', 'tag': '150f3a3b-28b5-435a-b826-0eee61f6b143', 'status': 'completed', 'code': 200} May 31 11:56:13.154205 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99040) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 11:56:13.199923 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port c495e266-af80-4662-84fa-21d967aef128 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:13.205083 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port c495e266-af80-4662-84fa-21d967aef128 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:13.205462 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:13.240643 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c238b699-e488-49b4-b530-9c34724d8753 req-e9b5e7e6-3748-41bc-8100-d0078f308844 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:13.243736 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-e9b5e7e6-3748-41bc-8100-d0078f308844 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 1097 time: 0.1674573 May 31 11:56:13.341523 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-cdabd71d-f63a-4aec-910d-2e7a270597aa service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=150f3a3b-28b5-435a-b826-0eee61f6b143 HTTP/1.1" status: 200 len: 217 time: 0.0742390 May 31 11:56:13.359599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e5e8ddd-f8c2-410c-962a-40ee4edbbbec req-16df06aa-1da9-4fb0-a2c5-31fd6aecc14f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:13.360288 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-4e5e8ddd-f8c2-410c-962a-40ee4edbbbec req-16df06aa-1da9-4fb0-a2c5-31fd6aecc14f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:56:13.387578 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:13.408693 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-d24028ff-647d-4ddf-9403-491151af4988 None None] Provisioning complete for port c495e266-af80-4662-84fa-21d967aef128 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:13.409001 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-d24028ff-647d-4ddf-9403-491151af4988 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:13.432186 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c238b699-e488-49b4-b530-9c34724d8753 req-b5e0573e-575c-467b-a50d-efeb354077f6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:13.433974 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-b5e0573e-575c-467b-a50d-efeb354077f6 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 200 len: 855 time: 0.0697773 May 31 11:56:13.457363 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ae1952a-a4e0-423a-a760-c367d40b76d8 req-1a082034-54ba-41f2-8a88-f96d8f1b9704 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:13.458887 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ae1952a-a4e0-423a-a760-c367d40b76d8 req-1a082034-54ba-41f2-8a88-f96d8f1b9704 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1854551 May 31 11:56:13.470385 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:56:13.475984 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port c495e266-af80-4662-84fa-21d967aef128, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=c495e266-af80-4662-84fa-21d967aef128,segment=NetworkSegment(0eba2510-55a0-4815-bcb1-2dc142aabf11),segment_id=0eba2510-55a0-4815-bcb1-2dc142aabf11)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:13.481263 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Sync routers for ids ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:56:13.511486 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Routers {'4c61aefe-0df0-474e-8c1a-3fa776e16bcc'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:56:13.512609 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:55:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:13.529308 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ae1952a-a4e0-423a-a760-c367d40b76d8 req-34220dfc-b049-4280-9543-2aede726db8f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0569105 May 31 11:56:13.544252 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:13.548852 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-4a8b2014-a16c-45d0-8004-336760731c2a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:13.579427 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-4e5e8ddd-f8c2-410c-962a-40ee4edbbbec req-16df06aa-1da9-4fb0-a2c5-31fd6aecc14f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:56:13.586238 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4e5e8ddd-f8c2-410c-962a-40ee4edbbbec req-16df06aa-1da9-4fb0-a2c5-31fd6aecc14f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 944 time: 0.3929610 May 31 11:56:13.606484 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-67a291e1-32df-42b8-a157-cd1f0a3b7f84 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=709626b6-807b-4256-9031-1578948f1fe8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1585925 May 31 11:56:13.611607 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-d24028ff-647d-4ddf-9403-491151af4988 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:13.629969 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Notify callbacks [] for router, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:13.633641 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 926 time: 3.8137641 May 31 11:56:13.641153 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 59986) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:13.677079 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=c495e266-af80-4662-84fa-21d967aef128,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:13.685914 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:13.708224 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.db.api [None req-d24028ff-647d-4ddf-9403-491151af4988 None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99041) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 31 11:56:13.708619 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_db.api [None req-d24028ff-647d-4ddf-9403-491151af4988 None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 31 11:56:13.757126 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Set router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:56:13.801339 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4e5e8ddd-f8c2-410c-962a-40ee4edbbbec req-53e9c4d2-601f-4212-b91f-7e85115cb931 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2142408 May 31 11:56:13.935087 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-ed7fc5d2-dbbf-4eaa-a184-f83c88d98dc4 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3197107 May 31 11:56:14.060861 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:56:12Z', 'revision_number': 3, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:11Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:14.064649 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Checking router: 4c61aefe-0df0-474e-8c1a-3fa776e16bcc for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:56:14.064999 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Updating router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc port 4ccc4832-093a-44f0-a0bb-60f43b576309 binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:56:14.138418 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:14.246948 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c238b699-e488-49b4-b530-9c34724d8753 req-57546355-3c02-40f4-94c1-736042cd87e9 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3010597 May 31 11:56:14.470183 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device c495e266-af80-4662-84fa-21d967aef128 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:14.541751 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port c495e266-af80-4662-84fa-21d967aef128 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:14.542245 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:14.622467 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:14.639714 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Transition to ACTIVE for port object 4ccc4832-093a-44f0-a0bb-60f43b576309 will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:14.663719 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-65dc58f7-45c7-4285-8b9e-44514b89c53d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:14.665173 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-65dc58f7-45c7-4285-8b9e-44514b89c53d service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 1097 time: 0.2592793 May 31 11:56:14.719918 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:14.731825 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:14.806429 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Attempting to bind port 4ccc4832-093a-44f0-a0bb-60f43b576309 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:56:14.806801 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Attempting to bind port 4ccc4832-093a-44f0-a0bb-60f43b576309 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:56:14.807215 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Attempting to bind port 4ccc4832-093a-44f0-a0bb-60f43b576309 on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:14.817406 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 55, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:56:14.818004 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:56:14.818233 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:14.818665 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:56:14.819651 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Bound port: 4ccc4832-093a-44f0-a0bb-60f43b576309, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:56:14.820029 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:14.850944 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3a01ef52-7e9b-461a-aff0-aacfd1416eff None None] Pushing event updated for resources: {'Port': ['ID=4ccc4832-093a-44f0-a0bb-60f43b576309,revision_number=2']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:14.853929 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-3a01ef52-7e9b-461a-aff0-aacfd1416eff None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:14.899423 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:14.910996 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] For port 4ccc4832-093a-44f0-a0bb-60f43b576309, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:56:14.917553 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] For port 4ccc4832-093a-44f0-a0bb-60f43b576309, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=4ccc4832-093a-44f0-a0bb-60f43b576309,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:56:14.944128 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-4e214d18-d09b-4817-ab3d-33e36ad6cdad service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:14.945540 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-4e214d18-d09b-4817-ab3d-33e36ad6cdad service neutron] 10.222.0.47 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 916 time: 0.2692795 May 31 11:56:14.953886 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b38a8f50-7a5a-4a4d-bfb4-be3f7b31ae2b req-88cfdab5-1096-4275-ad54-d8d24f767660 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:14.955250 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b38a8f50-7a5a-4a4d-bfb4-be3f7b31ae2b req-88cfdab5-1096-4275-ad54-d8d24f767660 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.2522881 May 31 11:56:15.012336 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:15.030130 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.004s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:15.069263 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a0cb00fc-ebb4-44df-a0c5-8de736e72540 req-f6c6c135-e944-41e1-aa1f-8638bc1e3e7e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:15.079360 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a0cb00fc-ebb4-44df-a0c5-8de736e72540 req-f6c6c135-e944-41e1-aa1f-8638bc1e3e7e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 944 time: 0.1429973 May 31 11:56:15.138538 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a0cb00fc-ebb4-44df-a0c5-8de736e72540 req-61db4bc0-348b-4c97-ac2a-f54cbad1c2e7 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0575628 May 31 11:56:15.176226 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-febf2c21-09a5-48df-8c0e-e3785c091167 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=150f3a3b-28b5-435a-b826-0eee61f6b143 HTTP/1.1" status: 200 len: 217 time: 0.0893812 May 31 11:56:15.181381 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Routers {'4c61aefe-0df0-474e-8c1a-3fa776e16bcc'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:56:15.183427 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:55:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:15.261086 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b38a8f50-7a5a-4a4d-bfb4-be3f7b31ae2b req-65000e6d-8ca3-4e77-a747-638ce733b03c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1262310 May 31 11:56:15.290243 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3a01ef52-7e9b-461a-aff0-aacfd1416eff None None] Pushing event updated for resources: {'Port': ['ID=4ccc4832-093a-44f0-a0bb-60f43b576309,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:15.294874 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-3a01ef52-7e9b-461a-aff0-aacfd1416eff None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.265s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:15.326446 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-d1bbbe50-c418-4b4e-994a-c84ad37c62f6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:15.327846 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-d1bbbe50-c418-4b4e-994a-c84ad37c62f6 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 200 len: 855 time: 0.1418922 May 31 11:56:15.403502 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] Set router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:56:15.415793 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-93027c97-daba-4902-83d7-195a800469fb service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=709626b6-807b-4256-9031-1578948f1fe8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0738497 May 31 11:56:15.528900 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Allocated IP 172.20.0.1 (87442cb7-bfc4-4bf2-9d82-09a2c5a5344f/c01a063a-2236-4017-91be-4828f1866f28/5c78fca6-f530-48c3-8190-890cb005d8f5) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:56:15.660360 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-2d441a9a-b580-420c-9a4d-331e4e70c976 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2285376 May 31 11:56:15.714781 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:56:12Z', 'revision_number': 3, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:14Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:15.718888 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b19cc5f2-b48e-4575-b022-d1d3f9833255 None None] The sync data for ids ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'] in dv-cinder-791239-4-iscsi is: [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:56:12Z', 'revision_number': 3, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:14Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:56:15.801881 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:15.833297 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Transition to ACTIVE for port object 5c78fca6-f530-48c3-8190-890cb005d8f5 will not be triggered until provisioned by entity DHCP. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:15.858581 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6aaded36-ad75-4a2f-a0fa-bcb750ef4e6e req-b9c631f0-1d3a-4886-b6e4-ea21b4d16fef service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1907065 May 31 11:56:15.931820 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:15.935809 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:16.015902 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:16.019009 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:16.025498 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:56:16.054235 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Pushing event updated for resources: {'Port': ['ID=5c78fca6-f530-48c3-8190-890cb005d8f5,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:16.061029 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:16.102608 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:16.386730 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:16.427576 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd62c278-40ee-4a61-ba89-8535a873274e req-9d28d02a-0e91-4628-bcb8-edee03a40d45 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:16.429712 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fd62c278-40ee-4a61-ba89-8535a873274e req-9d28d02a-0e91-4628-bcb8-edee03a40d45 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 944 time: 0.1642063 May 31 11:56:16.440121 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-aeb69759-f52b-4913-97ee-8e30cd32ad60 None None] Provisioning for port 5c78fca6-f530-48c3-8190-890cb005d8f5 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:16.450829 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-aeb69759-f52b-4913-97ee-8e30cd32ad60 None None] Provisioning complete for port 5c78fca6-f530-48c3-8190-890cb005d8f5 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:16.451213 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-aeb69759-f52b-4913-97ee-8e30cd32ad60 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:16.503799 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fd62c278-40ee-4a61-ba89-8535a873274e req-79749d4c-21ca-4bef-9e9d-48236c605250 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0632105 May 31 11:56:16.575186 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-aeb69759-f52b-4913-97ee-8e30cd32ad60 None None] Port 5c78fca6-f530-48c3-8190-890cb005d8f5 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:56:16.578406 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 4ccc4832-093a-44f0-a0bb-60f43b576309 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:16.751550 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c570c9bd-7d0c-4082-8d39-6869bb4960d4 req-b8446610-3079-4e36-bccc-00cff26d6ac4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:16.755018 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c570c9bd-7d0c-4082-8d39-6869bb4960d4 req-b8446610-3079-4e36-bccc-00cff26d6ac4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.3372993 May 31 11:56:16.800141 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 4ccc4832-093a-44f0-a0bb-60f43b576309 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:16.806879 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 4ccc4832-093a-44f0-a0bb-60f43b576309 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:16.807254 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:16.820925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 60058) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:16.916519 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c570c9bd-7d0c-4082-8d39-6869bb4960d4 req-31017902-7dcf-4c64-ad68-6fecf28c6297 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1461515 May 31 11:56:16.949497 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:16.984694 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.002339 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 4ccc4832-093a-44f0-a0bb-60f43b576309, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=4ccc4832-093a-44f0-a0bb-60f43b576309,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:17.006888 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.024189 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:17.027938 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Usage tracker for resource:security_group_rule and tenant:369e884ff01b452d8d12f364cb43e72d is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:17.030861 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:17.036422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:17.051469 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.079690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:17.102780 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.104753 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:17.163594 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Pushing event updated for resources: {'Port': ['ID=5c78fca6-f530-48c3-8190-890cb005d8f5,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:17.166062 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.170679 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:17.182816 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=4ccc4832-093a-44f0-a0bb-60f43b576309,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:17.187629 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:17.202267 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.359527 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.368648 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-41394f23-aa64-42de-bb0e-87a0e39f9c90 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:17.391869 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:17.393969 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:17.411615 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=369e884ff01b452d8d12f364cb43e72d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5893552 May 31 11:56:17.418363 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 60070) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:17.419288 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d6755a4d-69c5-4bd2-a98e-743c6b0f69c7,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:17.425094 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:17.487617 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a9466766-27e4-4adf-bb37-9aa162ece57f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.518238 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a9466766-27e4-4adf-bb37-9aa162ece57f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.607702 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:56:17.686309 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a9466766-27e4-4adf-bb37-9aa162ece57f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.696762 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0258e847-60e8-43f1-95d1-9535d1f89785 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:17.703917 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f771b187-4811-4ff5-8ba5-f377cb2a44e0 None None] Provisioning complete for port 5c78fca6-f530-48c3-8190-890cb005d8f5 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:17.704500 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f771b187-4811-4ff5-8ba5-f377cb2a44e0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.712516 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a9466766-27e4-4adf-bb37-9aa162ece57f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d6755a4d-69c5-4bd2-a98e-743c6b0f69c7,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:17.722960 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a9466766-27e4-4adf-bb37-9aa162ece57f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:17.722960 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a9466766-27e4-4adf-bb37-9aa162ece57f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:17.722960 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a9466766-27e4-4adf-bb37-9aa162ece57f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/d6755a4d-69c5-4bd2-a98e-743c6b0f69c7 HTTP/1.1" status: 204 len: 168 time: 0.2999353 May 31 11:56:17.785037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b037151d-65c1-4dd0-a882-7a9fe3df17e9 req-102db4f2-1058-4c14-8475-34ad6cf41e22 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:17.790342 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f771b187-4811-4ff5-8ba5-f377cb2a44e0 None None] Port 5c78fca6-f530-48c3-8190-890cb005d8f5 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:56:17.795517 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b037151d-65c1-4dd0-a882-7a9fe3df17e9 req-102db4f2-1058-4c14-8475-34ad6cf41e22 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 944 time: 0.1514778 May 31 11:56:17.824359 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:56:17.839277 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Sync routers for ids ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:56:17.853688 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] 10.222.0.48 "PUT /v2.0/routers/4c61aefe-0df0-474e-8c1a-3fa776e16bcc/add_router_interface HTTP/1.1" status: 200 len: 503 time: 4.2001705 May 31 11:56:17.878460 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Routers {'4c61aefe-0df0-474e-8c1a-3fa776e16bcc'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:56:17.879134 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:55:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:17.889864 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b037151d-65c1-4dd0-a882-7a9fe3df17e9 req-116d27ab-6c3b-4a04-a1af-6f4616d43a23 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0932825 May 31 11:56:17.989333 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 150f3a3b-28b5-435a-b826-0eee61f6b143 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:56:18.191898 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Set router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:56:18.237900 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad9a128e-e237-495f-9271-6d0282e16e94 req-a2c48ca0-2197-4e7b-a34b-ccf129928243 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:18.239846 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ad9a128e-e237-495f-9271-6d0282e16e94 req-a2c48ca0-2197-4e7b-a34b-ccf129928243 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1335380 May 31 11:56:18.292829 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ad9a128e-e237-495f-9271-6d0282e16e94 req-4d1ea291-197a-4b2d-9626-070c263b9d16 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0456390 May 31 11:56:18.393986 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 150f3a3b-28b5-435a-b826-0eee61f6b143, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=150f3a3b-28b5-435a-b826-0eee61f6b143,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:18.456705 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '150f3a3b-28b5-435a-b826-0eee61f6b143', 'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'port_id': '150f3a3b-28b5-435a-b826-0eee61f6b143', 'mac_address': 'fa:16:3e:76:45:72', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 356, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c58bc8a-d411-461e-a483-39773ff0d6f4', 'ip_address': '172.20.0.10'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:56:18.471710 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 4ccc4832-093a-44f0-a0bb-60f43b576309 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:18.606247 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 60108) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:18.610056 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:18.631980 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:18.649081 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:18.653400 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Usage tracker for resource:security_group_rule and tenant:f38754868f58427c94258a9bb201edfb is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:18.657081 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:18.663742 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:18.672060 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 150f3a3b-28b5-435a-b826-0eee61f6b143 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:56:18.700980 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 150f3a3b-28b5-435a-b826-0eee61f6b143, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=150f3a3b-28b5-435a-b826-0eee61f6b143,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:18.701924 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 4ccc4832-093a-44f0-a0bb-60f43b576309 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:18.702300 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:18.767873 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:18.785120 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:18.787909 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:18.821254 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '5c78fca6-f530-48c3-8190-890cb005d8f5', 'name': '', 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'mac_address': 'fa:16:3e:a0:6e:c8', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_interface', 'standard_attr_id': 222, 'fixed_ips': [{'subnet_id': 'c01a063a-2236-4017-91be-4828f1866f28', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:15Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 2, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'subnets': [{'id': 'c01a063a-2236-4017-91be-4828f1866f28', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:18.851882 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:18.860945 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-738cbd82-0f50-443d-8a6a-9d16fb79977a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:18.876414 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Checking router: 4c61aefe-0df0-474e-8c1a-3fa776e16bcc for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:56:18.876780 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Updating router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc port 5c78fca6-f530-48c3-8190-890cb005d8f5 binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:56:18.905940 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=f38754868f58427c94258a9bb201edfb&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2983258 May 31 11:56:18.907862 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9e484e92-82a2-422f-a2ee-3be0bbe55623,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:18.909178 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60126) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:18.916606 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.916964 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.917417 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.917671 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.918229 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.919597 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.920277 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.920771 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.921384 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.922626 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.923131 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.923613 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:18.928031 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:18.972698 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=150f3a3b-28b5-435a-b826-0eee61f6b143,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:18.978950 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.191s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:18.979885 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:19.047871 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d904f782-979d-4619-922b-6b9e343c46a3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:19.115484 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d904f782-979d-4619-922b-6b9e343c46a3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:19.164903 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['150f3a3b-28b5-435a-b826-0eee61f6b143'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:56:19.282990 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b512166-e11b-45bf-bf06-e6ed308e9cba req-b281738f-2c51-4544-97e4-184314113ccf tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:19.284731 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3b512166-e11b-45bf-bf06-e6ed308e9cba req-b281738f-2c51-4544-97e4-184314113ccf tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 945 time: 0.2511523 May 31 11:56:19.365532 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3b512166-e11b-45bf-bf06-e6ed308e9cba req-d9234848-0e26-47f5-939b-a351c6aaead8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0719037 May 31 11:56:19.429543 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d904f782-979d-4619-922b-6b9e343c46a3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:19.436727 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:19.474923 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d904f782-979d-4619-922b-6b9e343c46a3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9e484e92-82a2-422f-a2ee-3be0bbe55623,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:19.485543 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d904f782-979d-4619-922b-6b9e343c46a3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:19.487968 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d904f782-979d-4619-922b-6b9e343c46a3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:19.490028 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d904f782-979d-4619-922b-6b9e343c46a3 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/9e484e92-82a2-422f-a2ee-3be0bbe55623 HTTP/1.1" status: 204 len: 168 time: 0.5793581 May 31 11:56:19.650041 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17dfa2d4-3cda-4821-898d-bd2479b1322a req-6fbb165a-1ab3-4b85-bcc9-5e8baa353af8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:19.651495 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-17dfa2d4-3cda-4821-898d-bd2479b1322a req-6fbb165a-1ab3-4b85-bcc9-5e8baa353af8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1232927 May 31 11:56:19.736571 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-17dfa2d4-3cda-4821-898d-bd2479b1322a req-8d8f0ee8-3486-461d-897d-70db99d33c58 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0719941 May 31 11:56:19.813957 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:19.844608 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Transition to ACTIVE for port object 5c78fca6-f530-48c3-8190-890cb005d8f5 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:19.961018 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:19.972389 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5821890d-78aa-444f-b8f1-404627382df9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:20.066716 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d0d9436-e352-4dfc-861c-5722119e5ef9 None None] Pushing event updated for resources: {'Port': ['ID=5c78fca6-f530-48c3-8190-890cb005d8f5,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:20.126799 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5d0d9436-e352-4dfc-861c-5722119e5ef9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:20.280459 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Attempting to bind port 5c78fca6-f530-48c3-8190-890cb005d8f5 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:56:20.280917 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Attempting to bind port 5c78fca6-f530-48c3-8190-890cb005d8f5 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:56:20.282045 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Attempting to bind port 5c78fca6-f530-48c3-8190-890cb005d8f5 on network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:20.289189 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 55, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:56:20.290332 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Checking segment: {'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:56:20.290746 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:20.291152 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Bound using segment: {'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:56:20.292270 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Bound port: 5c78fca6-f530-48c3-8190-890cb005d8f5, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '0eba2510-55a0-4815-bcb1-2dc142aabf11', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 376, 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:56:20.292660 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:20.420442 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] For port 5c78fca6-f530-48c3-8190-890cb005d8f5, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:56:20.424148 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] For port 5c78fca6-f530-48c3-8190-890cb005d8f5, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=5c78fca6-f530-48c3-8190-890cb005d8f5,segment=NetworkSegment(0eba2510-55a0-4815-bcb1-2dc142aabf11),segment_id=0eba2510-55a0-4815-bcb1-2dc142aabf11)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:56:20.519148 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60172) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:20.529530 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:20.540343 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:20.644027 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Routers {'4c61aefe-0df0-474e-8c1a-3fa776e16bcc'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:56:20.644975 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:55:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:20.731237 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3284196a-03ed-4364-825e-58b646103d6e req-ce65e1bf-c0e4-4f8b-8214-73d3c6022f9c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:20.732927 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3284196a-03ed-4364-825e-58b646103d6e req-ce65e1bf-c0e4-4f8b-8214-73d3c6022f9c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 945 time: 0.1609690 May 31 11:56:20.860409 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3284196a-03ed-4364-825e-58b646103d6e req-8c464f7c-9a31-4714-b81a-d3d526b0a34e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1200545 May 31 11:56:20.912337 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d0d9436-e352-4dfc-861c-5722119e5ef9 None None] Pushing event updated for resources: {'Port': ['ID=5c78fca6-f530-48c3-8190-890cb005d8f5,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:20.914606 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:20.923897 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5d0d9436-e352-4dfc-861c-5722119e5ef9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.384s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:20.951109 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:20.958627 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Usage tracker for resource:security_group_rule and tenant:b628d8e2960e4d68a3a1a7864ae89e8c is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:20.966754 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:20.978690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:21.088136 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b5f8c09c-28cf-48f4-8c1e-dea9f685065a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:21.092407 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] Set router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:56:21.129248 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e53195d3-490f-4b1b-a238-0f35740b4514 req-5502183b-38b4-42ed-945d-5006155cb79d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:21.130786 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e53195d3-490f-4b1b-a238-0f35740b4514 req-5502183b-38b4-42ed-945d-5006155cb79d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1227241 May 31 11:56:21.255547 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e53195d3-490f-4b1b-a238-0f35740b4514 req-af292a32-f783-45fd-8c1f-fc594d656432 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1048801 May 31 11:56:21.255547 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:21.440004 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:21.456163 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1c072cef-6505-4ec1-b26c-527cdef678a3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:21.533664 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=b628d8e2960e4d68a3a1a7864ae89e8c&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0127251 May 31 11:56:21.536338 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 60196) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:21.536631 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7179a758-8e0a-47d3-b000-76cd302bfd5f,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:21.559712 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.570287 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.571026 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.571762 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.572516 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '5c78fca6-f530-48c3-8190-890cb005d8f5', 'name': '', 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'mac_address': 'fa:16:3e:a0:6e:c8', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_interface', 'standard_attr_id': 222, 'fixed_ips': [{'subnet_id': 'c01a063a-2236-4017-91be-4828f1866f28', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:15Z', 'updated_at': '2021-05-31T11:56:20Z', 'revision_number': 4, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'subnets': [{'id': 'c01a063a-2236-4017-91be-4828f1866f28', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:56:21.573238 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.575909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.576803 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.577490 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.578331 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.578902 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.579460 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.580022 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:21.586118 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:21.593458 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e0cc8891-6932-4888-b2c2-938f1e01e566 None None] The sync data for ids ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'] in dv-cinder-791239-4-iscsi is: [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '5c78fca6-f530-48c3-8190-890cb005d8f5', 'name': '', 'network_id': '87442cb7-bfc4-4bf2-9d82-09a2c5a5344f', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'mac_address': 'fa:16:3e:a0:6e:c8', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_interface', 'standard_attr_id': 222, 'fixed_ips': [{'subnet_id': 'c01a063a-2236-4017-91be-4828f1866f28', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:15Z', 'updated_at': '2021-05-31T11:56:20Z', 'revision_number': 4, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'subnets': [{'id': 'c01a063a-2236-4017-91be-4828f1866f28', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:56:21.691183 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-02b4bcb2-21b2-4086-8558-d62c34bbce2f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:21.732279 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-02b4bcb2-21b2-4086-8558-d62c34bbce2f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:21.908814 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-02b4bcb2-21b2-4086-8558-d62c34bbce2f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:21.922629 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b9543422-119f-4f2e-9abc-90e52694e716 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:21.939585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-02b4bcb2-21b2-4086-8558-d62c34bbce2f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7179a758-8e0a-47d3-b000-76cd302bfd5f,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:21.943036 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-02b4bcb2-21b2-4086-8558-d62c34bbce2f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:21.978098 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 150f3a3b-28b5-435a-b826-0eee61f6b143 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:21.979117 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-02b4bcb2-21b2-4086-8558-d62c34bbce2f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:21.981814 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-02b4bcb2-21b2-4086-8558-d62c34bbce2f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/7179a758-8e0a-47d3-b000-76cd302bfd5f HTTP/1.1" status: 204 len: 168 time: 0.4347086 May 31 11:56:22.146049 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8973089c-4ef9-46b6-931f-3fd05d19aad7 req-a33d2f90-d931-42c7-befb-d08849f19fb4 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:22.149109 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8973089c-4ef9-46b6-931f-3fd05d19aad7 req-a33d2f90-d931-42c7-befb-d08849f19fb4 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 945 time: 0.1387522 May 31 11:56:22.208168 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8973089c-4ef9-46b6-931f-3fd05d19aad7 req-ba7c14dc-394f-4bb3-9135-e91220631470 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0526981 May 31 11:56:22.242335 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 150f3a3b-28b5-435a-b826-0eee61f6b143 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:22.249838 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 150f3a3b-28b5-435a-b826-0eee61f6b143 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:22.249838 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:22.454480 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:22.514481 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '231d5bcd-ca61-435d-8029-4d0537a194ad', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '150f3a3b-28b5-435a-b826-0eee61f6b143'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:56:22.517067 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-a10914f8-2e27-4494-98f3-204603148207" -d '{"events": [{"server_uuid": "231d5bcd-ca61-435d-8029-4d0537a194ad", "name": "network-vif-plugged", "status": "completed", "tag": "150f3a3b-28b5-435a-b826-0eee61f6b143"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:56:22.541665 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 150f3a3b-28b5-435a-b826-0eee61f6b143, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=150f3a3b-28b5-435a-b826-0eee61f6b143,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:22.592456 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 5c78fca6-f530-48c3-8190-890cb005d8f5 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:22.698017 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:56:22 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-b020c900-272a-417b-a316-713794ef8d6e x-openstack-request-id: req-b020c900-272a-417b-a316-713794ef8d6e {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:56:22.698352 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "231d5bcd-ca61-435d-8029-4d0537a194ad", "name": "network-vif-plugged", "status": "completed", "tag": "150f3a3b-28b5-435a-b826-0eee61f6b143", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:56:22.699315 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-b020c900-272a-417b-a316-713794ef8d6e {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:56:22.700942 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '231d5bcd-ca61-435d-8029-4d0537a194ad', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '150f3a3b-28b5-435a-b826-0eee61f6b143', 'code': 200} May 31 11:56:22.730446 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:22.815920 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75059ff3-2b95-41a2-b87d-7a66c8815799 req-055165c9-313b-4899-a76e-0ff7686ffbd8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:22.821709 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75059ff3-2b95-41a2-b87d-7a66c8815799 req-055165c9-313b-4899-a76e-0ff7686ffbd8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1829877 May 31 11:56:22.873189 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5821890d-78aa-444f-b8f1-404627382df9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:22.971272 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-75059ff3-2b95-41a2-b87d-7a66c8815799 req-f9ed20c9-8468-42cb-a9c2-b370b2f6c650 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1483223 May 31 11:56:23.124224 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=150f3a3b-28b5-435a-b826-0eee61f6b143,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:23.135053 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.262s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:23.241783 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 5c78fca6-f530-48c3-8190-890cb005d8f5 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:23.247463 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 5c78fca6-f530-48c3-8190-890cb005d8f5 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:23.247938 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:23.287422 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4699f00-511e-488e-a2d9-920d96a480a8 req-c2b89307-02cd-4791-a6c3-1b63f26f5fa5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:23.287422 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f4699f00-511e-488e-a2d9-920d96a480a8 req-c2b89307-02cd-4791-a6c3-1b63f26f5fa5 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 1099 time: 0.1692328 May 31 11:56:23.405127 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f4699f00-511e-488e-a2d9-920d96a480a8 req-201800aa-0985-4dea-bff6-bb4882bc8eb5 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1122692 May 31 11:56:23.624648 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea8b526c-a560-469c-9b95-20031d601f5d req-f77dbdec-fc76-42e4-8a6c-6a677fabc203 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:23.630518 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ea8b526c-a560-469c-9b95-20031d601f5d req-f77dbdec-fc76-42e4-8a6c-6a677fabc203 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 946 time: 0.2231908 May 31 11:56:23.666726 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8c065a03-c59b-4de6-b135-0da3bb78728a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:23.670316 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:23.673661 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:23.763656 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ea8b526c-a560-469c-9b95-20031d601f5d req-d36757e2-c461-450f-8c9a-d8baa031fac8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1137955 May 31 11:56:23.770349 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 5c78fca6-f530-48c3-8190-890cb005d8f5, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=5c78fca6-f530-48c3-8190-890cb005d8f5,segment=NetworkSegment(0eba2510-55a0-4815-bcb1-2dc142aabf11),segment_id=0eba2510-55a0-4815-bcb1-2dc142aabf11)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:23.845437 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:23.847837 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:23.968517 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=5c78fca6-f530-48c3-8190-890cb005d8f5,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:23.972888 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:24.220302 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbb227f7-7e31-4ffc-972d-be441b8b646d req-f8a4f3eb-3da8-47fa-8cb5-46d433364ab1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:24.224042 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fbb227f7-7e31-4ffc-972d-be441b8b646d req-f8a4f3eb-3da8-47fa-8cb5-46d433364ab1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1050072 May 31 11:56:24.282176 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fbb227f7-7e31-4ffc-972d-be441b8b646d req-802514e4-a40c-43cc-8caa-ea911dba779b tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0527780 May 31 11:56:24.466486 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 5c78fca6-f530-48c3-8190-890cb005d8f5 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:56:24.548635 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 5c78fca6-f530-48c3-8190-890cb005d8f5 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:24.550546 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:25.239772 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:25.694398 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb57bf35-7ef6-49ec-871f-a8daa3a87c17 req-c3df3deb-17c9-4093-8e37-75b4ea827be8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:25.707116 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60296) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:25.710193 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cb57bf35-7ef6-49ec-871f-a8daa3a87c17 req-c3df3deb-17c9-4093-8e37-75b4ea827be8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 946 time: 0.1958742 May 31 11:56:25.766808 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cb57bf35-7ef6-49ec-871f-a8daa3a87c17 req-47a62033-7220-4b8d-8d33-b8678742876b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0551956 May 31 11:56:25.862974 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:25.881883 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:25.887340 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Usage tracker for resource:security_group_rule and tenant:49fb401a2705492686e4af9b39a595a4 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:25.891229 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:25.898417 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:26.035426 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:26.074546 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3fd432cf-3086-4575-b15a-5b01038ecb7d req-723780ed-2e0b-4f4d-9db9-28a967f09a34 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:26.085548 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3fd432cf-3086-4575-b15a-5b01038ecb7d req-723780ed-2e0b-4f4d-9db9-28a967f09a34 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.2054224 May 31 11:56:26.096829 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:26.127798 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:26.138126 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:26.166125 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3fd432cf-3086-4575-b15a-5b01038ecb7d req-be3862d5-2aa1-45f0-b3df-4cd4f99e9e76 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0716600 May 31 11:56:26.217120 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=49fb401a2705492686e4af9b39a595a4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5059538 May 31 11:56:26.217746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60308) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:26.229725 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0da4b751-5ffc-4aca-82a6-ecb15cee63cd,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:26.234247 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:26.288519 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d916456c-fffa-4384-aed2-eafc7488b956 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:26.317783 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d916456c-fffa-4384-aed2-eafc7488b956 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:26.514126 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d916456c-fffa-4384-aed2-eafc7488b956 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:26.518661 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:26.532527 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d916456c-fffa-4384-aed2-eafc7488b956 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0da4b751-5ffc-4aca-82a6-ecb15cee63cd,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:26.535247 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d916456c-fffa-4384-aed2-eafc7488b956 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:26.544067 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d916456c-fffa-4384-aed2-eafc7488b956 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:26.547775 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d916456c-fffa-4384-aed2-eafc7488b956 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/0da4b751-5ffc-4aca-82a6-ecb15cee63cd HTTP/1.1" status: 204 len: 168 time: 0.3267107 May 31 11:56:27.673351 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1aa4a9cf-de6a-4248-935e-0c109629f1fd req-94ad0a3d-3b5d-44c5-bafe-33dda5857b3c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:27.680883 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1aa4a9cf-de6a-4248-935e-0c109629f1fd req-94ad0a3d-3b5d-44c5-bafe-33dda5857b3c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 946 time: 0.1125348 May 31 11:56:27.699881 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a64769e-700d-4b8c-90f4-a933ef8c5d58 req-34c5ae92-3097-4447-87bf-8ada9d0b0822 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:27.707525 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3a64769e-700d-4b8c-90f4-a933ef8c5d58 req-34c5ae92-3097-4447-87bf-8ada9d0b0822 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1174314 May 31 11:56:27.759196 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1aa4a9cf-de6a-4248-935e-0c109629f1fd req-5045cfd9-861c-4668-b09f-551afa966b24 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0750482 May 31 11:56:27.792234 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3a64769e-700d-4b8c-90f4-a933ef8c5d58 req-bcd0a86b-4979-4e8c-816c-fab17e7c2caf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0838542 May 31 11:56:30.476464 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:56:31.356724 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-562b92df-e84f-465b-b678-31ba89760ba8 req-b842d2bb-8679-453b-9ebf-5dff0a5f8f9d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:31.362948 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-562b92df-e84f-465b-b678-31ba89760ba8 req-b842d2bb-8679-453b-9ebf-5dff0a5f8f9d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 946 time: 0.1032946 May 31 11:56:31.422645 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-562b92df-e84f-465b-b678-31ba89760ba8 req-c7d07254-27a4-4d13-8336-67b23bd543a7 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0578227 May 31 11:56:31.429836 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aadcd4c4-6e92-49a9-9faa-5b59f979978f req-fea1b01e-6228-4bc8-b5b3-b372e1546199 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:31.431618 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-aadcd4c4-6e92-49a9-9faa-5b59f979978f req-fea1b01e-6228-4bc8-b5b3-b372e1546199 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1321430 May 31 11:56:31.549216 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-aadcd4c4-6e92-49a9-9faa-5b59f979978f req-303e1d82-f1f4-440e-878b-133129def5f5 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1105340 May 31 11:56:32.749228 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b8772f1-46dd-4af3-8644-2c5951cec529 req-2d8fdb41-e2b3-482b-b140-48021ba43cee tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:32.751597 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2b8772f1-46dd-4af3-8644-2c5951cec529 req-2d8fdb41-e2b3-482b-b140-48021ba43cee tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0844338 May 31 11:56:32.800912 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2b8772f1-46dd-4af3-8644-2c5951cec529 req-43678919-2ef1-4fbc-96bd-90c99b0e69ea tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0437448 May 31 11:56:34.014639 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f263a0a-8a62-41f6-a08f-6ed449b5fff0 req-80085173-5d0f-41d9-a5de-c382bdcf708c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:34.018031 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6f263a0a-8a62-41f6-a08f-6ed449b5fff0 req-80085173-5d0f-41d9-a5de-c382bdcf708c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1011894 May 31 11:56:34.063614 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6f263a0a-8a62-41f6-a08f-6ed449b5fff0 req-723300f8-5d0d-4ac0-913c-1168966be89f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0411332 May 31 11:56:34.359390 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-561255e7-6ddc-41c7-b385-3a7c573ffafc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:34.360849 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-561255e7-6ddc-41c7-b385-3a7c573ffafc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/networks?id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb HTTP/1.1" status: 200 len: 814 time: 0.2358599 May 31 11:56:34.377637 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-bdce5867-5170-40dc-9a6e-da38051bf6fb tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/quotas/39ae37364a184ae7ae341ea7b2847c35 HTTP/1.1" status: 200 len: 375 time: 0.0098875 May 31 11:56:34.500529 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-0f1c8b49-458e-4016-888b-be94771fa77a tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Synchronizing usage tracker for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:56:34.665114 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-0f1c8b49-458e-4016-888b-be94771fa77a tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Unset dirty status for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:56:34.666722 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-0f1c8b49-458e-4016-888b-be94771fa77a tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=39ae37364a184ae7ae341ea7b2847c35&fields=id HTTP/1.1" status: 200 len: 302 time: 0.2816136 May 31 11:56:34.816061 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-95a234a9-6541-4754-af00-47dc8e3ea489 service neutron] 10.222.0.48 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1379199 May 31 11:56:34.952723 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-60f40005-fe74-4b4a-83c4-ea0b573951ec service neutron] 10.222.0.48 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1285644 May 31 11:56:35.103309 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60468) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:35.273269 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:35.309146 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:35.314411 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Usage tracker for resource:security_group_rule and tenant:7b1a92e2eca84f5881efb48d22514cba is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:35.320090 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:35.339835 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:35.522406 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6d6c9fbc-9f16-49ad-ba0f-80fd9c17b28e req-dc74d7b9-4d58-4501-b6f8-f71350005d68 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 210 time: 0.0558977 May 31 11:56:35.607048 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f02a5c67-18b2-49d1-9013-2782c3b15211 req-11c11842-7188-4eb2-a25d-95cdcae04e31 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:35.609943 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f02a5c67-18b2-49d1-9013-2782c3b15211 req-11c11842-7188-4eb2-a25d-95cdcae04e31 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1099 time: 0.1383474 May 31 11:56:35.616471 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7bdea9b0-79e9-4850-9371-1614170833c8 req-7d497369-cd34-43c8-aa12-df37b0d9a1d7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:35.617905 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7bdea9b0-79e9-4850-9371-1614170833c8 req-7d497369-cd34-43c8-aa12-df37b0d9a1d7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.2411010 May 31 11:56:35.622704 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:35.690299 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f02a5c67-18b2-49d1-9013-2782c3b15211 req-d597aba0-0db1-4a3c-b9bb-abf2b5ba72fd service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0713978 May 31 11:56:35.697619 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7bdea9b0-79e9-4850-9371-1614170833c8 req-49122253-b3df-44d9-8e24-847c2cd640fa tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0619273 May 31 11:56:35.754102 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:35.762992 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:35.805785 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=7b1a92e2eca84f5881efb48d22514cba&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6999414 May 31 11:56:35.806526 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=84739a10-8ebd-4010-ae1a-800ee6d8b335,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:35.808295 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 60498) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:35.809748 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:35.873224 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1d09fc9c-3092-40a8-a175-ee5eb4d1253f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:35.908326 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1d09fc9c-3092-40a8-a175-ee5eb4d1253f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:36.102475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1d09fc9c-3092-40a8-a175-ee5eb4d1253f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:36.107225 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ae4fb2c2-77f4-42a8-8b8b-c8425c843ec0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:36.120518 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1d09fc9c-3092-40a8-a175-ee5eb4d1253f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=84739a10-8ebd-4010-ae1a-800ee6d8b335,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:36.123267 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1d09fc9c-3092-40a8-a175-ee5eb4d1253f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:36.124168 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1d09fc9c-3092-40a8-a175-ee5eb4d1253f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:36.125316 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-1d09fc9c-3092-40a8-a175-ee5eb4d1253f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/84739a10-8ebd-4010-ae1a-800ee6d8b335 HTTP/1.1" status: 204 len: 168 time: 0.3157735 May 31 11:56:36.628439 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3254556c-effa-4332-a463-9e9bf5318ea6 req-49a977c2-2f94-43f7-ab2b-3aca19e2d849 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 210 time: 0.0521739 May 31 11:56:36.917477 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7b39e84-2746-4140-941e-2bc833b1af6d req-a3a8f2d6-5eff-48dc-b9f2-b493c4b362be tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:36.919032 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a7b39e84-2746-4140-941e-2bc833b1af6d req-a3a8f2d6-5eff-48dc-b9f2-b493c4b362be tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0883062 May 31 11:56:36.968163 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a7b39e84-2746-4140-941e-2bc833b1af6d req-785cc255-ca99-4595-ae98-84e264717267 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0431077 May 31 11:56:37.083446 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 60534) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:37.102306 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:37.116815 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:37.120672 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Usage tracker for resource:security_group_rule and tenant:b24b092270c5457d99b8f68dee724882 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:37.123551 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:37.129261 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:37.219109 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:37.337101 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:37.352476 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:37.395192 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=b24b092270c5457d99b8f68dee724882&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3086421 May 31 11:56:37.395503 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60546) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:37.402047 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8d5d5a50-8cf6-4e50-a202-c7006a639ae9,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:37.422688 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:37.514926 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-320e2fff-f12b-45bb-9950-30d8566048dd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:37.547946 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-320e2fff-f12b-45bb-9950-30d8566048dd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:37.569054 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-9cd99dec-9261-410f-b2da-c40caa333d4b service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1375852 May 31 11:56:37.714428 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-320e2fff-f12b-45bb-9950-30d8566048dd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:37.723156 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:37.746960 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-320e2fff-f12b-45bb-9950-30d8566048dd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8d5d5a50-8cf6-4e50-a202-c7006a639ae9,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:37.758763 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-320e2fff-f12b-45bb-9950-30d8566048dd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:37.786357 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-320e2fff-f12b-45bb-9950-30d8566048dd admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:37.793463 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-320e2fff-f12b-45bb-9950-30d8566048dd admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/8d5d5a50-8cf6-4e50-a202-c7006a639ae9 HTTP/1.1" status: 204 len: 168 time: 0.3966389 May 31 11:56:37.812723 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-22d66cb3-b83c-4392-be2e-784bee4b80ce service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:37.814537 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-22d66cb3-b83c-4392-be2e-784bee4b80ce service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1099 time: 0.1889608 May 31 11:56:37.850770 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0db1b4b9-b905-41bb-b8a6-779443cd5b6b req-4f3889ce-5dbe-44f1-8ca6-e92acd0a7d7f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 210 time: 0.0883365 May 31 11:56:38.019143 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acff052e-cdd0-4ab2-92dc-2177f6149de1 req-30ceb056-181a-45bf-8dca-6bef6ff1b511 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:38.020589 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acff052e-cdd0-4ab2-92dc-2177f6149de1 req-30ceb056-181a-45bf-8dca-6bef6ff1b511 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1982028 May 31 11:56:38.027514 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-bc7eaa2e-7e0f-4de4-963a-a32122880947 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:38.028873 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-bc7eaa2e-7e0f-4de4-963a-a32122880947 service neutron] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 920 time: 0.1693668 May 31 11:56:38.136228 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-dcf9e123-97b3-4b0f-b2ef-c88943a99210 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778 HTTP/1.1" status: 200 len: 217 time: 0.0905073 May 31 11:56:38.162770 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acff052e-cdd0-4ab2-92dc-2177f6149de1 req-22fa212c-fb6d-49c5-b022-07116e0af648 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1327784 May 31 11:56:38.171630 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c88bc35a-3ed2-41e2-aa1f-3f0f2cbc9c37 req-ece6bee4-c2ce-4eef-ab9d-a5b787d440f4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:38.180655 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c88bc35a-3ed2-41e2-aa1f-3f0f2cbc9c37 req-ece6bee4-c2ce-4eef-ab9d-a5b787d440f4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0974557 May 31 11:56:38.244008 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-2b7bc1f2-edca-4090-8021-07c5d242af7a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:38.245524 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-2b7bc1f2-edca-4090-8021-07c5d242af7a service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.0863323 May 31 11:56:38.246059 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c88bc35a-3ed2-41e2-aa1f-3f0f2cbc9c37 req-42514fba-71e9-468c-bf21-19d2d73ae190 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0643599 May 31 11:56:38.297016 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-063257ff-9606-4eb0-9bd0-2a51870f1c20 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0423937 May 31 11:56:38.463749 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-3487cd44-392c-4bac-900e-a3c1d3ff96e7 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1559205 May 31 11:56:38.699888 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9aff4f77-4540-436c-803a-47434ed6dd21 req-d85dc25f-bcdf-4432-9031-88a62992f238 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2145698 May 31 11:56:38.968687 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-86a00e79-8342-476e-b6c9-a1b284050722 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:38.970295 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-86a00e79-8342-476e-b6c9-a1b284050722 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1099 time: 0.1158113 May 31 11:56:39.024956 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-dba08352-522e-4960-94eb-75421e9e4726 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:39.026520 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-dba08352-522e-4960-94eb-75421e9e4726 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.47 "GET /v2.0/networks?id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb HTTP/1.1" status: 200 len: 814 time: 0.2268558 May 31 11:56:39.028933 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:39.035415 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb805e92-3991-469f-aa0c-56e3dbc932c5 req-43e07752-006c-4577-a1d5-6c50b9dbcf30 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:39.043435 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-cb805e92-3991-469f-aa0c-56e3dbc932c5 req-43e07752-006c-4577-a1d5-6c50b9dbcf30 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.2107084 May 31 11:56:39.064711 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'port': {'device_id': '175ced20-c6a2-4bec-82db-394e58bd641e', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'admin_state_up': True, 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:56:39.173255 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-03033c1e-97a8-4feb-924d-b1a32cca2f39 req-cc0a8c0d-4b72-43b5-b8a6-388e8ec84e6a tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 210 time: 0.1991882 May 31 11:56:39.181754 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-cb805e92-3991-469f-aa0c-56e3dbc932c5 req-e8e0e805-cde7-4610-9d68-7812f644a501 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1371245 May 31 11:56:39.300268 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-bb11b684-b1cc-4391-a879-e5e1b1d7e845 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:39.301730 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-bb11b684-b1cc-4391-a879-e5e1b1d7e845 service neutron] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 920 time: 0.2187507 May 31 11:56:39.312929 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:39.325214 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:39.404268 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-dd6c5a2c-2239-4e8a-a61e-f8e0ef6a8803 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778 HTTP/1.1" status: 200 len: 217 time: 0.0951908 May 31 11:56:39.414550 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Made reservation on behalf of 39ae37364a184ae7ae341ea7b2847c35 for: {'port': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:56:39.417804 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:39.487986 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-7f01d5cf-11b1-4989-85e9-8eef90a74d9f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:39.491816 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-7f01d5cf-11b1-4989-85e9-8eef90a74d9f service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.0753839 May 31 11:56:39.554094 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b3e83995-4f6a-49a7-90c5-93cf1dd93fea req-df724bdc-5aea-4e44-8a3f-b131e0599fed tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:39.562575 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-0199a3da-d1b6-449c-b316-728c8918fc6c service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0519791 May 31 11:56:39.565997 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b3e83995-4f6a-49a7-90c5-93cf1dd93fea req-df724bdc-5aea-4e44-8a3f-b131e0599fed tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.2197750 May 31 11:56:39.671199 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b3e83995-4f6a-49a7-90c5-93cf1dd93fea req-70885c35-d25a-4df1-8f86-9a5231083892 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1042368 May 31 11:56:39.727154 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Allocated IP 172.20.0.4 (0fae71a7-8e60-4f7d-a467-822e3d8d87fb/672ec900-b55c-44a0-9ed9-7e27beeace12/3c46c8f8-777e-4ff2-8577-976c595eec9e) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:56:39.775244 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-3448ec71-425f-4bed-adf5-eebe03da0945 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1920230 May 31 11:56:39.993711 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-31cce4ed-720a-4bae-b8c1-7217c730108b req-876a0f7a-98ff-4cb4-b273-ec0ea7a628be service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2107244 May 31 11:56:40.003978 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:40.022359 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Transition to ACTIVE for port object 3c46c8f8-777e-4ff2-8577-976c595eec9e will not be triggered until provisioned by entity DHCP. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:40.165481 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:40.171337 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:40.264348 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:56:40.267914 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:56:40.291441 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:56:40.402704 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'Port': ['ID=3c46c8f8-777e-4ff2-8577-976c595eec9e,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:40.419136 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.248s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:40.510611 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:40.520615 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:40.576841 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd92d179-d640-4e5f-b00e-12c13896444c req-a00135a5-cfb8-4d9c-b357-07e319129933 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:40.585044 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd92d179-d640-4e5f-b00e-12c13896444c req-a00135a5-cfb8-4d9c-b357-07e319129933 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.2550220 May 31 11:56:40.640251 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.5939233 May 31 11:56:40.656726 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fd92d179-d640-4e5f-b00e-12c13896444c req-80c89266-d974-474a-ace7-53dcb8ac23dc tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0703394 May 31 11:56:40.736316 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Request body: {'port': {'device_id': '175ced20-c6a2-4bec-82db-394e58bd641e', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:56:40.867277 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-6ef91203-c086-4e06-97e9-d582398fc8a2 None None] Provisioning for port 3c46c8f8-777e-4ff2-8577-976c595eec9e completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:56:40.871933 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-6ef91203-c086-4e06-97e9-d582398fc8a2 None None] Provisioning complete for port 3c46c8f8-777e-4ff2-8577-976c595eec9e triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:56:40.872492 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6ef91203-c086-4e06-97e9-d582398fc8a2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:40.961856 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:41.036486 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-6ef91203-c086-4e06-97e9-d582398fc8a2 None None] Port 3c46c8f8-777e-4ff2-8577-976c595eec9e cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:56:41.067466 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-647dc368-d632-4726-ac7c-f1e62ca9721e req-d182e9ed-78aa-48f2-a7ee-1fecca99b3cf tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:41.067466 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-647dc368-d632-4726-ac7c-f1e62ca9721e req-d182e9ed-78aa-48f2-a7ee-1fecca99b3cf tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Synchronizing usage tracker for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:56:41.104666 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2dc65c9b-6047-410c-a3bf-7f09be1c632e req-ef3b2c0f-f056-4f11-8ef1-e487ab7a9982 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:41.115884 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2dc65c9b-6047-410c-a3bf-7f09be1c632e req-ef3b2c0f-f056-4f11-8ef1-e487ab7a9982 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1617646 May 31 11:56:41.191019 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2dc65c9b-6047-410c-a3bf-7f09be1c632e req-d57065fa-ffd2-48c1-b8a9-d6700b61c3fd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0737143 May 31 11:56:41.198840 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-647dc368-d632-4726-ac7c-f1e62ca9721e req-d182e9ed-78aa-48f2-a7ee-1fecca99b3cf tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Unset dirty status for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:56:41.211808 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-647dc368-d632-4726-ac7c-f1e62ca9721e req-d182e9ed-78aa-48f2-a7ee-1fecca99b3cf tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 931 time: 0.3326266 May 31 11:56:41.373092 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-647dc368-d632-4726-ac7c-f1e62ca9721e req-cf109b30-3c24-4b94-87d4-2d4eb3e13c17 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1602769 May 31 11:56:41.666266 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 3c46c8f8-777e-4ff2-8577-976c595eec9e {{(pid=99039) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:56:41.675966 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:41.712358 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Transition to ACTIVE for port object 3c46c8f8-777e-4ff2-8577-976c595eec9e will not be triggered until provisioned by entity L2. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:56:41.794357 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:41.799548 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:41.888181 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7ef16ed-8560-48b4-b28b-72004b077027 req-9cf120fc-45de-4926-9230-689235e6840f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:41.889726 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c7ef16ed-8560-48b4-b28b-72004b077027 req-9cf120fc-45de-4926-9230-689235e6840f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1069307 May 31 11:56:41.953256 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c7ef16ed-8560-48b4-b28b-72004b077027 req-31baf8a4-b3c4-43e1-90f0-434ead1aa320 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0564969 May 31 11:56:42.002708 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Pushing event updated for resources: {'Port': ['ID=3c46c8f8-777e-4ff2-8577-976c595eec9e,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:42.005844 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.206s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:42.114114 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Attempting to bind port 3c46c8f8-777e-4ff2-8577-976c595eec9e on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:56:42.114338 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Attempting to bind port 3c46c8f8-777e-4ff2-8577-976c595eec9e by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:56:42.114667 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Attempting to bind port 3c46c8f8-777e-4ff2-8577-976c595eec9e on network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:42.124086 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Port 3c46c8f8-777e-4ff2-8577-976c595eec9e on network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:56:42.124412 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Attempting to bind port 3c46c8f8-777e-4ff2-8577-976c595eec9e on network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:56:42.131174 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 56, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:56:42.135493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Checking segment: {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:56:42.135493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Bound using segment: {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:56:42.135493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Bound port: 3c46c8f8-777e-4ff2-8577-976c595eec9e, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '061f4d79-99bf-47ba-a95e-2ecb79298c69', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 366, 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb'}}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:56:42.135493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:42.246956 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] For port 3c46c8f8-777e-4ff2-8577-976c595eec9e, host hv-cinder-79123, cleared binding levels {{(pid=99039) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:56:42.253607 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] For port 3c46c8f8-777e-4ff2-8577-976c595eec9e, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99039) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:56:42.351367 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:42.356961 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:42.437187 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:42.437324 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:42.438649 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] 10.222.0.47 "PUT /v2.0/ports/3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 1093 time: 1.7070498 May 31 11:56:42.450810 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:56:42.467702 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-a6f0bde5-5e1f-4563-8362-82f041a118d3" -d '{"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:56:42.510016 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Pushing event updated for resources: {'Port': ['ID=3c46c8f8-777e-4ff2-8577-976c595eec9e,revision_number=3']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:42.513405 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:42.522122 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fa459d19-5b28-4472-8860-6c2b8a7a5d21 req-e5128d71-cbde-496e-aa27-17d84a6e4a7d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:42.528694 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fa459d19-5b28-4472-8860-6c2b8a7a5d21 req-e5128d71-cbde-496e-aa27-17d84a6e4a7d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.2211003 May 31 11:56:42.629747 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fa459d19-5b28-4472-8860-6c2b8a7a5d21 req-32b55d59-4183-4702-995e-6d3c482a7f6f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0965710 May 31 11:56:42.758714 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:56:42 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-b897b302-8cdf-4536-ad3e-6db53e5f11cf x-openstack-request-id: req-b897b302-8cdf-4536-ad3e-6db53e5f11cf {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:56:42.759306 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:56:42.759536 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-b897b302-8cdf-4536-ad3e-6db53e5f11cf {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:56:42.760055 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'status': 'completed', 'code': 200} May 31 11:56:42.809021 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4afe0c8f-0c65-4aff-abdd-79e99a646914 req-3cf12d79-797a-4cf0-9d77-c900dfe42309 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:42.817298 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4afe0c8f-0c65-4aff-abdd-79e99a646914 req-3cf12d79-797a-4cf0-9d77-c900dfe42309 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1908100 May 31 11:56:42.862009 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-4afe0c8f-0c65-4aff-abdd-79e99a646914 req-edea4dd5-22bf-4b41-90c1-8ae343b8000e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Synchronizing usage tracker for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:56:43.010815 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-4afe0c8f-0c65-4aff-abdd-79e99a646914 req-edea4dd5-22bf-4b41-90c1-8ae343b8000e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Unset dirty status for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:56:43.015166 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4afe0c8f-0c65-4aff-abdd-79e99a646914 req-edea4dd5-22bf-4b41-90c1-8ae343b8000e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1932836 May 31 11:56:43.173375 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-12bee72f-3145-4ac9-9a53-50cb70df4636 req-c7381087-6573-4c11-bd20-8bc0e276e412 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:43.175001 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-12bee72f-3145-4ac9-9a53-50cb70df4636 req-c7381087-6573-4c11-bd20-8bc0e276e412 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0862620 May 31 11:56:43.265754 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-12bee72f-3145-4ac9-9a53-50cb70df4636 req-bfd7d34c-a5ad-440d-8cd4-263d9fa34cf1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0775008 May 31 11:56:43.858758 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dfbf991d-eb3b-41a7-9738-0ef7edf36d22 req-eb863e8c-49b2-4ff5-9cc2-905c19b3aa48 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:43.860228 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dfbf991d-eb3b-41a7-9738-0ef7edf36d22 req-eb863e8c-49b2-4ff5-9cc2-905c19b3aa48 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0899432 May 31 11:56:43.911859 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dfbf991d-eb3b-41a7-9738-0ef7edf36d22 req-4edea400-b67a-4011-a72d-99fdaa2fcb8c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0456760 May 31 11:56:44.239235 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0efd1c78-ce5b-456c-a6e1-3d91b8ee0871 req-acbace3f-fb62-40e0-9a7a-d6de54e0293b tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:44.240978 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0efd1c78-ce5b-456c-a6e1-3d91b8ee0871 req-acbace3f-fb62-40e0-9a7a-d6de54e0293b tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1184103 May 31 11:56:44.309677 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0efd1c78-ce5b-456c-a6e1-3d91b8ee0871 req-eec86e0d-c044-4dd0-a0a3-4da912068799 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0624604 May 31 11:56:44.549035 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50c19d7b-9512-4c21-aa93-ce8f3c82ae9c req-3e36e4e6-299e-4d4f-b900-ce8a0c4f2264 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:44.553114 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-50c19d7b-9512-4c21-aa93-ce8f3c82ae9c req-3e36e4e6-299e-4d4f-b900-ce8a0c4f2264 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1343765 May 31 11:56:44.635419 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-50c19d7b-9512-4c21-aa93-ce8f3c82ae9c req-6324746d-0572-4aad-a480-33b954a72415 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0762434 May 31 11:56:45.125452 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97b82a27-e6ab-4348-801e-e7cfbf2458a4 req-44d212e2-bdbb-409c-9ffe-d1e2a276dcbc tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:45.127979 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-97b82a27-e6ab-4348-801e-e7cfbf2458a4 req-44d212e2-bdbb-409c-9ffe-d1e2a276dcbc tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1004410 May 31 11:56:45.181354 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-97b82a27-e6ab-4348-801e-e7cfbf2458a4 req-a69ad349-5929-4622-b5e4-1a19fefd8184 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0473249 May 31 11:56:45.554069 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4164b23c-9dde-4bc8-8201-19e58aab1bbe req-ff3114f6-e072-404f-954c-4407053137ac tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:45.555599 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4164b23c-9dde-4bc8-8201-19e58aab1bbe req-ff3114f6-e072-404f-954c-4407053137ac tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1263640 May 31 11:56:45.608778 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4164b23c-9dde-4bc8-8201-19e58aab1bbe req-919e45db-36cb-4eb1-8bd9-ba349c5fed41 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0473793 May 31 11:56:45.839063 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d19eb1d8-5955-4f3f-9e2f-6abaac641b5a req-14121751-76c3-44d1-9348-de9150c2eaa4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:45.840587 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d19eb1d8-5955-4f3f-9e2f-6abaac641b5a req-14121751-76c3-44d1-9348-de9150c2eaa4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0904274 May 31 11:56:45.888706 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d19eb1d8-5955-4f3f-9e2f-6abaac641b5a req-19240864-700b-44cc-98b9-5188c88aca42 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0418935 May 31 11:56:46.561888 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0bbc23ae-7150-4ee8-a18c-15e8f63a2f57 req-500d891c-957f-41ab-bd57-f12c0eff6df8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:46.563338 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0bbc23ae-7150-4ee8-a18c-15e8f63a2f57 req-500d891c-957f-41ab-bd57-f12c0eff6df8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0870173 May 31 11:56:46.613155 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0bbc23ae-7150-4ee8-a18c-15e8f63a2f57 req-bb8a1028-9bed-4695-b4c7-570ce1aab89e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0440044 May 31 11:56:46.830355 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-698e3ae5-627a-4895-b99a-5a0577592195 req-aa3f369e-e6be-4248-82fe-2739212d071d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:46.836334 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-698e3ae5-627a-4895-b99a-5a0577592195 req-aa3f369e-e6be-4248-82fe-2739212d071d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.0929449 May 31 11:56:46.900614 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-698e3ae5-627a-4895-b99a-5a0577592195 req-9ef5481e-f283-433f-86a3-4dc4d6ec0a77 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0603740 May 31 11:56:47.102590 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e531656-7d5a-4a22-8716-f953b9a0faef req-374bd6ca-f63e-4f19-b69b-d00fbd65dca6 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:47.105487 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9e531656-7d5a-4a22-8716-f953b9a0faef req-374bd6ca-f63e-4f19-b69b-d00fbd65dca6 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0830488 May 31 11:56:47.148832 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9e531656-7d5a-4a22-8716-f953b9a0faef req-68da40d6-d1df-4add-b48a-b4ff5632dbfc tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0391450 May 31 11:56:47.987999 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-032d3ba9-9eb5-4d94-9c59-2cbad12d31e1 req-d27a52f5-bf3b-41e6-8412-7d8cb55676ee tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:47.989532 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-032d3ba9-9eb5-4d94-9c59-2cbad12d31e1 req-d27a52f5-bf3b-41e6-8412-7d8cb55676ee tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1057453 May 31 11:56:48.042373 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-032d3ba9-9eb5-4d94-9c59-2cbad12d31e1 req-802c5e19-ad75-49e8-abbc-62694ad87e43 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0456159 May 31 11:56:48.122555 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-655a621c-314c-4307-9c40-1d3613c963c4 req-ed15d183-4e44-4518-a99f-93acbb783486 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:48.124004 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-655a621c-314c-4307-9c40-1d3613c963c4 req-ed15d183-4e44-4518-a99f-93acbb783486 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1044178 May 31 11:56:48.194385 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-655a621c-314c-4307-9c40-1d3613c963c4 req-f84e5eee-6a03-4b71-9fb3-9dfa3a317cae tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0645332 May 31 11:56:48.357110 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28abbb58-9421-47da-a753-d3e736b2aa80 req-43b4346e-0b84-47b2-ac1b-16ac38e0267c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:48.358585 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-28abbb58-9421-47da-a753-d3e736b2aa80 req-43b4346e-0b84-47b2-ac1b-16ac38e0267c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0897522 May 31 11:56:48.405273 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-28abbb58-9421-47da-a753-d3e736b2aa80 req-a85a338a-bae0-4bc6-a5b9-4884992b52c5 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410464 May 31 11:56:49.244068 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-04449b58-cc0c-4168-b852-1adae5654dcd req-5fda748f-aa07-435e-bbec-d7ced0437277 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:49.245535 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-04449b58-cc0c-4168-b852-1adae5654dcd req-5fda748f-aa07-435e-bbec-d7ced0437277 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0852973 May 31 11:56:49.295993 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-04449b58-cc0c-4168-b852-1adae5654dcd req-aa24bffe-10f4-4d64-8392-f9f6b51ca1b2 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0446143 May 31 11:56:49.382757 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b17f6730-b915-45e4-ab87-45e71a217c50 req-c2b25e02-3ec8-4ec4-97da-b3a90b4a677a tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:49.385490 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b17f6730-b915-45e4-ab87-45e71a217c50 req-c2b25e02-3ec8-4ec4-97da-b3a90b4a677a tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.0820842 May 31 11:56:49.432091 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b17f6730-b915-45e4-ab87-45e71a217c50 req-7544fa31-ca86-43b1-b63f-cb3f857aa839 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0423043 May 31 11:56:49.667222 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4c56d19-042d-4e11-a61a-3bf05562199b req-a7c65e1f-7352-4fe7-96bb-9ca09056b7a1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:49.668651 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e4c56d19-042d-4e11-a61a-3bf05562199b req-a7c65e1f-7352-4fe7-96bb-9ca09056b7a1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1497157 May 31 11:56:49.713938 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e4c56d19-042d-4e11-a61a-3bf05562199b req-3844af8d-93eb-470a-8846-f5b6f76b40d0 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390975 May 31 11:56:50.597335 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7835ced5-7351-4f1b-a452-3e1522f13ba1 req-c085ee5b-e6c7-4e70-9b62-c06923685b91 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:50.598960 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7835ced5-7351-4f1b-a452-3e1522f13ba1 req-c085ee5b-e6c7-4e70-9b62-c06923685b91 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1818490 May 31 11:56:50.664282 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7835ced5-7351-4f1b-a452-3e1522f13ba1 req-873cfe63-76bb-484b-b9df-a8994f154170 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0571289 May 31 11:56:50.704973 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e835cbf-d8ee-4f51-87c6-0eba8a69dd27 req-ab2e3e70-f884-45b9-a59d-e4dc71ede9a1 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:50.709032 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4e835cbf-d8ee-4f51-87c6-0eba8a69dd27 req-ab2e3e70-f884-45b9-a59d-e4dc71ede9a1 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1470101 May 31 11:56:50.774865 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4e835cbf-d8ee-4f51-87c6-0eba8a69dd27 req-87e6c0a0-2c61-453b-a702-d12e2965946c tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0553806 May 31 11:56:50.913977 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c93d27bc-8ac9-4f50-8928-c289081bc72b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:50.946172 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22a789d3-40e9-42e8-9f34-680e158741b5 req-71c571ec-9659-4f3a-9838-ed5b10832c77 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:50.947807 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-22a789d3-40e9-42e8-9f34-680e158741b5 req-71c571ec-9659-4f3a-9838-ed5b10832c77 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0939295 May 31 11:56:51.003252 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-22a789d3-40e9-42e8-9f34-680e158741b5 req-50883668-751e-4093-96ca-b76585ca263a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0497549 May 31 11:56:51.513975 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60892) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:51.634554 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:51.650732 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:51.655583 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Usage tracker for resource:security_group_rule and tenant:a0424338b350489eb36011912b7f4927 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:51.660294 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:51.668116 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:51.788016 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:51.874630 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:51.891972 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-301bde59-c93f-44fc-b608-d81ca19a2120 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.002s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:51.959460 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=a0424338b350489eb36011912b7f4927&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4436214 May 31 11:56:51.960786 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f3822929-a670-44e6-b7aa-74226cf26be8,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:51.963564 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60914) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:51.977213 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:52.001499 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b3c522a-ec4e-47c8-83f1-e3820da01ebc req-4ebaff2d-05b4-4376-9286-0b758fa2d23a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:52.009203 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4b3c522a-ec4e-47c8-83f1-e3820da01ebc req-4ebaff2d-05b4-4376-9286-0b758fa2d23a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1630261 May 31 11:56:52.037120 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-80da2f7d-642f-49fa-b014-1d26e540604c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:52.069096 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4b3c522a-ec4e-47c8-83f1-e3820da01ebc req-9b1b1419-3763-431f-a329-ebedbdb9d6c7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0588388 May 31 11:56:52.198328 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-80da2f7d-642f-49fa-b014-1d26e540604c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:52.277019 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e98834a-3985-4529-aac7-27fda4e36d65 req-7bd2a909-71ed-4d7f-a37a-b0a694e11978 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:52.284612 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4e98834a-3985-4529-aac7-27fda4e36d65 req-7bd2a909-71ed-4d7f-a37a-b0a694e11978 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.2453294 May 31 11:56:52.352481 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4e98834a-3985-4529-aac7-27fda4e36d65 req-18ffe294-2a88-4088-beac-1280ae629b70 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0669894 May 31 11:56:52.372109 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6b1023a-74de-4ac6-a4fd-6717782eb2f6 req-5a6120bd-5e67-4396-8dc2-efeaf159f8bf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:52.373705 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b6b1023a-74de-4ac6-a4fd-6717782eb2f6 req-5a6120bd-5e67-4396-8dc2-efeaf159f8bf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1621697 May 31 11:56:52.416453 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-80da2f7d-642f-49fa-b014-1d26e540604c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:52.421821 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e822ba5e-9880-4901-b35f-12efa0746228 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:52.424959 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b6b1023a-74de-4ac6-a4fd-6717782eb2f6 req-80437834-25e1-4bad-a0c6-b4a7ad24b20a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0452173 May 31 11:56:52.441471 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-80da2f7d-642f-49fa-b014-1d26e540604c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f3822929-a670-44e6-b7aa-74226cf26be8,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:52.447116 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-80da2f7d-642f-49fa-b014-1d26e540604c admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:52.448072 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-80da2f7d-642f-49fa-b014-1d26e540604c admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/f3822929-a670-44e6-b7aa-74226cf26be8 HTTP/1.1" status: 204 len: 168 time: 0.4835041 May 31 11:56:52.449024 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-80da2f7d-642f-49fa-b014-1d26e540604c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:53.100729 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 60940) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:53.119029 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.133526 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:53.137251 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Usage tracker for resource:security_group_rule and tenant:4daad8b79ffb46e2aed61c2e36cc6ea2 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:53.140094 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:53.146033 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:53.294109 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00df562b-5f34-45d5-8f0d-f66f458416cd req-ef518a0d-856c-4627-95b4-a3eee6df95a8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:53.295500 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-00df562b-5f34-45d5-8f0d-f66f458416cd req-ef518a0d-856c-4627-95b4-a3eee6df95a8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1229327 May 31 11:56:53.297149 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.346404 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-00df562b-5f34-45d5-8f0d-f66f458416cd req-bc6f4a07-66e1-407f-8d6b-d7d962b25bc1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424399 May 31 11:56:53.353415 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.393997 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.403451 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-2042e49b-ed4a-4835-8ebe-63a993b82dd6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:53.446775 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=4daad8b79ffb46e2aed61c2e36cc6ea2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3440211 May 31 11:56:53.454440 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ba34d5db-6bdb-459a-9d7b-1f05da15b6c3,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:53.454440 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:53.456682 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 60950) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:53.537054 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-379e75a9-503e-4673-ac63-9f2f668fafa2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.587449 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-df66d9b7-00ec-4786-88a2-215c99d11638 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.615506 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-df66d9b7-00ec-4786-88a2-215c99d11638 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.634691 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32c1c89e-fa38-4dd4-9a7d-bc15cc67541a req-2b08b57b-738c-4a94-aaad-9269c2927451 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:53.636157 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-32c1c89e-fa38-4dd4-9a7d-bc15cc67541a req-2b08b57b-738c-4a94-aaad-9269c2927451 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1544168 May 31 11:56:53.677080 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6f0bde5-5e1f-4563-8362-82f041a118d3 req-b2ed2855-335a-45be-b348-a28e78b11674 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:53.678871 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a6f0bde5-5e1f-4563-8362-82f041a118d3 req-b2ed2855-335a-45be-b348-a28e78b11674 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=39ae37364a184ae7ae341ea7b2847c35&device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1096 time: 0.1349180 May 31 11:56:53.693318 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ff0c3872-1f8f-4e28-803f-12ada7388cc5 req-cc14219c-7ef3-46c6-9362-2697a5dfd28c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:53.694788 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ff0c3872-1f8f-4e28-803f-12ada7388cc5 req-cc14219c-7ef3-46c6-9362-2697a5dfd28c tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1161718 May 31 11:56:53.705473 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-32c1c89e-fa38-4dd4-9a7d-bc15cc67541a req-abd8c7c6-b66f-4e34-91a7-2cb00f33497f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0573411 May 31 11:56:53.804642 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-df66d9b7-00ec-4786-88a2-215c99d11638 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.809806 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a9466766-27e4-4adf-bb37-9aa162ece57f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:53.824291 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df66d9b7-00ec-4786-88a2-215c99d11638 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ba34d5db-6bdb-459a-9d7b-1f05da15b6c3,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:53.827186 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-df66d9b7-00ec-4786-88a2-215c99d11638 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:53.832046 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ff0c3872-1f8f-4e28-803f-12ada7388cc5 req-e2b75e68-efa1-4d1d-badf-8b6ff218d5a9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1152449 May 31 11:56:53.843562 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-df66d9b7-00ec-4786-88a2-215c99d11638 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:53.844772 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-df66d9b7-00ec-4786-88a2-215c99d11638 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/ba34d5db-6bdb-459a-9d7b-1f05da15b6c3 HTTP/1.1" status: 204 len: 168 time: 0.3871577 May 31 11:56:53.969540 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6f0bde5-5e1f-4563-8362-82f041a118d3 req-9981a026-8d73-4c2d-a7bd-cd0f4049a651 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:53.971116 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a6f0bde5-5e1f-4563-8362-82f041a118d3 req-9981a026-8d73-4c2d-a7bd-cd0f4049a651 service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=39ae37364a184ae7ae341ea7b2847c35&shared=False HTTP/1.1" status: 200 len: 915 time: 0.2754939 May 31 11:56:54.118483 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6f0bde5-5e1f-4563-8362-82f041a118d3 req-6e282436-e998-4ed9-811b-798f9494fe50 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:54.119936 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a6f0bde5-5e1f-4563-8362-82f041a118d3 req-6e282436-e998-4ed9-811b-798f9494fe50 service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1412296 May 31 11:56:54.235390 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-c2deb48e-6693-4846-883f-db931150b4e3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:54.236885 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-c2deb48e-6693-4846-883f-db931150b4e3 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=39ae37364a184ae7ae341ea7b2847c35&device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1096 time: 0.0795884 May 31 11:56:54.316587 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-d0ed9804-55fe-4f61-8b12-94421805b815 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 217 time: 0.0647125 May 31 11:56:54.391861 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-f0230407-6f4d-4bf5-bbdd-eaba3c3d7125 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:54.393296 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-f0230407-6f4d-4bf5-bbdd-eaba3c3d7125 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=672ec900-b55c-44a0-9ed9-7e27beeace12 HTTP/1.1" status: 200 len: 853 time: 0.0661254 May 31 11:56:54.517491 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-40a1e1d4-cbb2-42ac-880f-1edabd92f221 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:54.519684 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-40a1e1d4-cbb2-42ac-880f-1edabd92f221 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1129050 May 31 11:56:54.591604 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-121918a7-6215-43cb-9128-88fc6aa4c474 req-d59a2708-a3a9-4c06-a68f-f0fdb85c8344 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:54.591604 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-121918a7-6215-43cb-9128-88fc6aa4c474 req-d59a2708-a3a9-4c06-a68f-f0fdb85c8344 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1127713 May 31 11:56:54.631609 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-121918a7-6215-43cb-9128-88fc6aa4c474 req-d53f3c11-dfb5-4b40-9428-0af248567be7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0395482 May 31 11:56:54.663395 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:54.689315 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-d57641af-0997-4044-a773-15b257da35a1 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1618791 May 31 11:56:54.895024 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38bc9f02-06a4-4557-a556-fc8b33248de1 req-a415681e-2d4d-428f-b91f-c7fa716f0845 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1964839 May 31 11:56:54.938153 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-af245d19-0867-4838-8dc0-ba5cbba9a0eb req-2d390f82-fcbb-4fe3-b95f-e70d8472a4a2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:54.955370 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-af245d19-0867-4838-8dc0-ba5cbba9a0eb req-2d390f82-fcbb-4fe3-b95f-e70d8472a4a2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1197553 May 31 11:56:55.008853 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-af245d19-0867-4838-8dc0-ba5cbba9a0eb req-27440ebb-7881-4e17-8569-b1ea60226b12 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0525239 May 31 11:56:55.099260 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-635519e7-5669-4dd0-a9dd-30c36743a474 req-8b9cc160-0100-497d-8bdf-abc5d5aa00be tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:55.102000 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-635519e7-5669-4dd0-a9dd-30c36743a474 req-8b9cc160-0100-497d-8bdf-abc5d5aa00be tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1039746 May 31 11:56:55.164918 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-635519e7-5669-4dd0-a9dd-30c36743a474 req-d6b1f397-5c7d-4023-badc-90655cfa5c25 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0535541 May 31 11:56:55.475765 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 32790) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:55.732976 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:55.750680 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:55.757325 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Usage tracker for resource:security_group_rule and tenant:e2771ea8dbc84080b85e5ca1ffabef6b is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:55.758058 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:55.763889 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:55.841726 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2061c810-f6b2-4906-9beb-13679d4935e1 req-91d912b3-27da-49aa-8560-59f0f9c27b3c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:55.843198 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2061c810-f6b2-4906-9beb-13679d4935e1 req-91d912b3-27da-49aa-8560-59f0f9c27b3c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0954423 May 31 11:56:55.865299 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:55.888751 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2061c810-f6b2-4906-9beb-13679d4935e1 req-ad926111-b3f7-46a1-94b1-56f81be0e1bf tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0395501 May 31 11:56:55.993937 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:56.008234 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7db20818-bb6b-4eaa-a60a-4c3ff1c3ba4d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:56.081768 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 32816) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:56.082367 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=e2771ea8dbc84080b85e5ca1ffabef6b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6052260 May 31 11:56:56.083725 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=617082f3-6f9e-48f1-bbb0-266ab5d84e39,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:56.088298 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:56.147903 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:56.236560 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4714ffb4-98fd-4acd-bb54-3cb391360752 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:56.273607 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4714ffb4-98fd-4acd-bb54-3cb391360752 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:56.303109 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd73bd93-bfa2-4f94-93e3-dc35a1f5be7a req-3612988b-40ab-434e-b7db-be3d7d9952cf tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:56.304717 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dd73bd93-bfa2-4f94-93e3-dc35a1f5be7a req-3612988b-40ab-434e-b7db-be3d7d9952cf tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1647897 May 31 11:56:56.365428 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dd73bd93-bfa2-4f94-93e3-dc35a1f5be7a req-dde8a053-a34e-49fd-9dce-2326cd264774 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0473924 May 31 11:56:56.450705 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4714ffb4-98fd-4acd-bb54-3cb391360752 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:56.455651 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:56.466871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4714ffb4-98fd-4acd-bb54-3cb391360752 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=617082f3-6f9e-48f1-bbb0-266ab5d84e39,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:56.469306 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4714ffb4-98fd-4acd-bb54-3cb391360752 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:56.475621 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4714ffb4-98fd-4acd-bb54-3cb391360752 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:56.477434 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-4714ffb4-98fd-4acd-bb54-3cb391360752 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/617082f3-6f9e-48f1-bbb0-266ab5d84e39 HTTP/1.1" status: 204 len: 168 time: 0.3934298 May 31 11:56:57.088237 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77b4a694-ca8c-4db5-96ae-043eac7f1d1b req-427c532d-e9d9-4bc1-977e-37316329be96 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:57.089747 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-77b4a694-ca8c-4db5-96ae-043eac7f1d1b req-427c532d-e9d9-4bc1-977e-37316329be96 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0879536 May 31 11:56:57.140249 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-77b4a694-ca8c-4db5-96ae-043eac7f1d1b req-5157a2a2-9f83-46a7-82b9-5d317b2b33cf tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0438225 May 31 11:56:57.211789 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 32852) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:57.228920 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:57.246198 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:56:57.247461 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Usage tracker for resource:security_group_rule and tenant:d165db6d9f504aa591e406510a245c89 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:56:57.251116 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:56:57.257667 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:56:57.346909 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:57.546536 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:57.559945 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1d84698e-26b7-4b1a-afff-b10a8b11359d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:57.592998 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c373fe6c-338a-44ba-a68b-1effac0dadb6 req-5d8ae09c-98f1-49d2-96a6-d6fdbb3dbbc3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:57.594951 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c373fe6c-338a-44ba-a68b-1effac0dadb6 req-5d8ae09c-98f1-49d2-96a6-d6fdbb3dbbc3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 943 time: 0.1022873 May 31 11:56:57.597320 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=d165db6d9f504aa591e406510a245c89&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3844762 May 31 11:56:57.598356 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e41d9738-01e4-4eba-ad57-72a0b8b4c1f3,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:57.600860 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 32868) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:56:57.611995 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:57.746812 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75fffc10-3761-428f-a12e-62f730de61f3 req-525e20b1-c526-49f4-b801-f5bd253a1bfe tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:57.748183 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-75fffc10-3761-428f-a12e-62f730de61f3 req-525e20b1-c526-49f4-b801-f5bd253a1bfe tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0883696 May 31 11:56:57.759242 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c373fe6c-338a-44ba-a68b-1effac0dadb6 req-661b0b98-17dc-4fd1-b40f-e4a2669a086d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1545131 May 31 11:56:57.777146 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-84488dbd-25a2-45bf-945f-b49a93d6a1fc admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:57.796193 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-75fffc10-3761-428f-a12e-62f730de61f3 req-3440a9ba-74ef-4b5c-bffa-376a769daa0d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0421088 May 31 11:56:57.804935 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-84488dbd-25a2-45bf-945f-b49a93d6a1fc admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:57.999246 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-84488dbd-25a2-45bf-945f-b49a93d6a1fc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:58.004769 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5dc04b6c-bcc3-4c7f-8b0a-17327a41864f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:58.020209 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-84488dbd-25a2-45bf-945f-b49a93d6a1fc admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e41d9738-01e4-4eba-ad57-72a0b8b4c1f3,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:58.022631 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-84488dbd-25a2-45bf-945f-b49a93d6a1fc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:58.034190 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-84488dbd-25a2-45bf-945f-b49a93d6a1fc admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:58.035294 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-84488dbd-25a2-45bf-945f-b49a93d6a1fc admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/e41d9738-01e4-4eba-ad57-72a0b8b4c1f3 HTTP/1.1" status: 204 len: 168 time: 0.4322562 May 31 11:56:58.149389 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 3c46c8f8-777e-4ff2-8577-976c595eec9e details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:56:58.343149 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 3c46c8f8-777e-4ff2-8577-976c595eec9e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:58.368368 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6898bdc2-711d-426d-a086-0ba6929db02a req-3487d48e-39a2-4445-b7ff-cb7b8249735c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:58.370273 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'port_id': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'mac_address': 'fa:16:3e:5c:72:1b', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 366, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:56:58.371193 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6898bdc2-711d-426d-a086-0ba6929db02a req-3487d48e-39a2-4445-b7ff-cb7b8249735c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0927444 May 31 11:56:58.435659 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:58.441842 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6898bdc2-711d-426d-a086-0ba6929db02a req-59ed1ae6-12c2-4067-ae70-2345a9ca13c9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0649080 May 31 11:56:58.466049 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 3c46c8f8-777e-4ff2-8577-976c595eec9e {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:56:58.484161 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 3c46c8f8-777e-4ff2-8577-976c595eec9e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:56:58.616711 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:56:58.618644 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-9b9c037d-7c1c-4d24-a218-2dcc73acea1e None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:56:58.683322 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=3c46c8f8-777e-4ff2-8577-976c595eec9e,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:56:58.690876 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.691211 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.691538 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.691868 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.692186 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.692503 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.693486 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.694051 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.694595 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.695148 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.695711 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.696264 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:56:58.701119 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:56:58.947142 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c93aaeb3-cc74-441c-aba2-af72861df372 req-c3e60cbe-1aba-4355-b6d3-9b92ea2b5177 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:58.950603 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c93aaeb3-cc74-441c-aba2-af72861df372 req-c3e60cbe-1aba-4355-b6d3-9b92ea2b5177 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 944 time: 0.0858898 May 31 11:56:58.998857 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c93aaeb3-cc74-441c-aba2-af72861df372 req-ea6791c7-e2f0-45d5-a4ca-8d51f44027b3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0445738 May 31 11:56:59.036382 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['3c46c8f8-777e-4ff2-8577-976c595eec9e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:56:59.663876 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e4a14a3-ef64-4a6a-95f4-42e9ae609ab7 req-318bd222-9b38-462e-8b6e-713a2e1f3246 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:56:59.665376 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8e4a14a3-ef64-4a6a-95f4-42e9ae609ab7 req-318bd222-9b38-462e-8b6e-713a2e1f3246 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0987420 May 31 11:56:59.710832 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8e4a14a3-ef64-4a6a-95f4-42e9ae609ab7 req-657e7a44-3a12-4ad5-b0a9-dab15aa402de tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393791 May 31 11:57:00.162735 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:57:00.164862 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 3c46c8f8-777e-4ff2-8577-976c595eec9e up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:57:00.200845 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ecef97d0-7f67-477f-ae10-5657ee1a3b85 req-cc2c518d-8b2c-4bc4-b9b7-93770cdf4637 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:00.202420 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ecef97d0-7f67-477f-ae10-5657ee1a3b85 req-cc2c518d-8b2c-4bc4-b9b7-93770cdf4637 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 944 time: 0.0878718 May 31 11:57:00.249738 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ecef97d0-7f67-477f-ae10-5657ee1a3b85 req-58d5d973-5798-49dc-9858-820c3291f45e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401838 May 31 11:57:00.368390 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:00.384841 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'port_id': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'mac_address': 'fa:16:3e:5a:6f:46', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 371, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:57:00.421728 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 3c46c8f8-777e-4ff2-8577-976c595eec9e completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:57:00.439954 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 3c46c8f8-777e-4ff2-8577-976c595eec9e triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:57:00.440299 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:00.493416 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:00.512831 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id e0a8cb19-42fc-4a14-b86a-efda3d732b98 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:57:00.535081 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:00.595232 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:00.617487 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:00.619860 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-87f8168a-c611-4d19-b7f7-39aedada206f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:00.657109 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:57:00.659403 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-79e20ae4-e7f2-4137-9fe4-03f6ceb2aea6" -d '{"events": [{"server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "name": "network-vif-plugged", "status": "completed", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:57:00.686040 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 3c46c8f8-777e-4ff2-8577-976c595eec9e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:00.709372 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=6']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:00.709372 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:57:00 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-c2e185ae-23f5-48c0-90b1-6b104cba0340 x-openstack-request-id: req-c2e185ae-23f5-48c0-90b1-6b104cba0340 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:57:00.709372 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "name": "network-vif-plugged", "status": "completed", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:57:00.709372 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-c2e185ae-23f5-48c0-90b1-6b104cba0340 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:57:00.709372 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'code': 200} May 31 11:57:00.709372 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:00.758780 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['e0a8cb19-42fc-4a14-b86a-efda3d732b98'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:57:00.797437 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:00.831279 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:00.920467 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=3c46c8f8-777e-4ff2-8577-976c595eec9e,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:00.924722 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:00.940463 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a68d8bc-74cb-42fc-8220-f6b681bd97a0 req-64444187-0941-4e82-936f-ca8bb9406c93 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:00.942358 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9a68d8bc-74cb-42fc-8220-f6b681bd97a0 req-64444187-0941-4e82-936f-ca8bb9406c93 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 944 time: 0.1138551 May 31 11:57:00.994309 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9a68d8bc-74cb-42fc-8220-f6b681bd97a0 req-7fd0f35d-96a9-48b2-8b48-d200a22fdb90 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0455761 May 31 11:57:01.138269 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b9233d0-6670-472f-a422-f42ce15823e0 req-19d76eea-cf38-4e37-a8ec-eaa11d0bbfed service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:01.139718 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3b9233d0-6670-472f-a422-f42ce15823e0 req-19d76eea-cf38-4e37-a8ec-eaa11d0bbfed service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1098 time: 0.0688002 May 31 11:57:01.183594 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3b9233d0-6670-472f-a422-f42ce15823e0 req-43c5e35c-2be2-4fc1-bb7c-9d3b8c537ec3 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0361574 May 31 11:57:01.450416 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34237584-7b99-4f69-992b-dac4cce33147 req-82870787-9ef5-44a9-a918-b1d94bb49487 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:01.450416 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-34237584-7b99-4f69-992b-dac4cce33147 req-82870787-9ef5-44a9-a918-b1d94bb49487 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.0819149 May 31 11:57:01.490219 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-34237584-7b99-4f69-992b-dac4cce33147 req-39d770c4-060e-4b8d-a816-9b7a3a8b0d98 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389152 May 31 11:57:02.119575 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33008) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:02.181361 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:57:02.263747 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port e0a8cb19-42fc-4a14-b86a-efda3d732b98 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:57:02.264126 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.280180 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3fffb2df-c545-4c6f-a6e2-605e1273c669 req-d31ad798-d72c-4aab-b214-869982316fe4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:02.286999 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3fffb2df-c545-4c6f-a6e2-605e1273c669 req-d31ad798-d72c-4aab-b214-869982316fe4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 944 time: 0.1687741 May 31 11:57:02.330585 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.353969 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3fffb2df-c545-4c6f-a6e2-605e1273c669 req-8347031d-c5e4-4a1e-a2c4-84dd06568196 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0584121 May 31 11:57:02.366703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:02.369401 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Usage tracker for resource:security_group_rule and tenant:3cfc0cbe13784454b20dd9abb108f576 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:02.372703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:02.379031 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:02.466046 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.490032 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.498727 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:02.547833 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.561954 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d916456c-fffa-4384-aed2-eafc7488b956 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:02.605086 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.606978 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:02.637410 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=3cfc0cbe13784454b20dd9abb108f576&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5165918 May 31 11:57:02.638259 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f89eb366-3205-4e84-af03-47ca44ea30e4,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:02.640217 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33020) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:02.641351 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:02.692346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=7']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:02.696167 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:02.705268 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5f2d374d-adef-4a4a-8a16-36f2600fb999 req-863e2fe1-b843-491b-8584-68bfb4c735c8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:02.706797 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5f2d374d-adef-4a4a-8a16-36f2600fb999 req-863e2fe1-b843-491b-8584-68bfb4c735c8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.1095307 May 31 11:57:02.708851 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:57:02.712436 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-64fe38bb-3272-44c8-8a67-6e0083566f38" -d '{"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-plugged", "status": "completed", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:57:02.712989 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ecdc24d6-8029-4b89-bc27-868a791d0d9d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.754042 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ecdc24d6-8029-4b89-bc27-868a791d0d9d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.768073 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5f2d374d-adef-4a4a-8a16-36f2600fb999 req-723d2cdd-d1e1-4243-9126-17ef5746df6a tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0517342 May 31 11:57:02.768395 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:57:02 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-184d8464-1168-4f81-b794-b84f25060e6e x-openstack-request-id: req-184d8464-1168-4f81-b794-b84f25060e6e {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:57:02.768698 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-plugged", "status": "completed", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:57:02.769075 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-184d8464-1168-4f81-b794-b84f25060e6e {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:57:02.769510 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'code': 200} May 31 11:57:02.906254 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ecdc24d6-8029-4b89-bc27-868a791d0d9d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.915974 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1d09fc9c-3092-40a8-a175-ee5eb4d1253f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:02.931818 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ecdc24d6-8029-4b89-bc27-868a791d0d9d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f89eb366-3205-4e84-af03-47ca44ea30e4,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:02.942416 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ecdc24d6-8029-4b89-bc27-868a791d0d9d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:02.948725 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ecdc24d6-8029-4b89-bc27-868a791d0d9d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:02.951839 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-ecdc24d6-8029-4b89-bc27-868a791d0d9d admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/f89eb366-3205-4e84-af03-47ca44ea30e4 HTTP/1.1" status: 204 len: 168 time: 0.3083060 May 31 11:57:03.002414 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-416dbc67-3125-4021-9d98-b6ece8f50cad req-48de00a9-4450-454a-ba7d-1ba88bdae6c5 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:03.005553 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-416dbc67-3125-4021-9d98-b6ece8f50cad req-48de00a9-4450-454a-ba7d-1ba88bdae6c5 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.1353128 May 31 11:57:03.075990 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-416dbc67-3125-4021-9d98-b6ece8f50cad req-63aad336-fde2-4631-83b7-8409b16313d3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0573864 May 31 11:57:03.543249 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-afc5e49e-243e-4813-b4f4-eb99a6fa5a97 req-dd71b1b4-822a-41b2-b461-51fe9483980e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:03.545411 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-afc5e49e-243e-4813-b4f4-eb99a6fa5a97 req-dd71b1b4-822a-41b2-b461-51fe9483980e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.1267314 May 31 11:57:03.643060 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-afc5e49e-243e-4813-b4f4-eb99a6fa5a97 req-a26729dc-9454-46b6-bc91-93892d3e9d5b tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0910723 May 31 11:57:03.726933 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4baf2771-55e8-4c47-9a45-09f7267dfede req-3f33613a-a722-42f0-bb8a-ad783c6bb1df tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:03.728511 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4baf2771-55e8-4c47-9a45-09f7267dfede req-3f33613a-a722-42f0-bb8a-ad783c6bb1df tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1261685 May 31 11:57:03.753191 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33066) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:03.776175 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4baf2771-55e8-4c47-9a45-09f7267dfede req-d56b5d80-52b6-4343-a4e9-69d56a69ad3e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0415146 May 31 11:57:03.810632 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:03.841402 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:03.848265 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Usage tracker for resource:security_group_rule and tenant:6ca1a7d55287446eb253c505ce2d789c is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:03.858133 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:03.863362 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:03.903380 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f06f5f73-9a00-4bd5-8b07-5db01cc499c9 req-48f12c51-f416-47a0-8018-2eb0608d1b1e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:03.904839 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f06f5f73-9a00-4bd5-8b07-5db01cc499c9 req-48f12c51-f416-47a0-8018-2eb0608d1b1e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1372151 May 31 11:57:03.982184 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f06f5f73-9a00-4bd5-8b07-5db01cc499c9 req-f4d830a2-a7de-4cdd-89f2-d485ff816c23 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0679438 May 31 11:57:04.031524 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:04.100252 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:04.118812 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:04.195990 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33082) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:04.208517 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=6ca1a7d55287446eb253c505ce2d789c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4538472 May 31 11:57:04.209599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6a3501d4-1763-48af-ae9d-02f80be44ef1,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:04.224704 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:04.276785 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-aec051dd-f0ab-4def-bede-b1267cc84f29 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:04.325879 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-aec051dd-f0ab-4def-bede-b1267cc84f29 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:04.490845 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-aec051dd-f0ab-4def-bede-b1267cc84f29 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:04.498483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8a36b70a-f9ea-4da2-b714-69389691f457 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:04.513327 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec051dd-f0ab-4def-bede-b1267cc84f29 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6a3501d4-1763-48af-ae9d-02f80be44ef1,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:04.516041 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-aec051dd-f0ab-4def-bede-b1267cc84f29 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:04.517436 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-aec051dd-f0ab-4def-bede-b1267cc84f29 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/6a3501d4-1763-48af-ae9d-02f80be44ef1 HTTP/1.1" status: 204 len: 168 time: 0.3203893 May 31 11:57:04.519126 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-aec051dd-f0ab-4def-bede-b1267cc84f29 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:05.068481 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0e460a1-1d74-4970-96b5-9713aaccd389 req-aa08368b-4f06-46b5-b4bd-d85a13bf715f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:05.076886 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0e460a1-1d74-4970-96b5-9713aaccd389 req-aa08368b-4f06-46b5-b4bd-d85a13bf715f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1451840 May 31 11:57:05.141197 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0e460a1-1d74-4970-96b5-9713aaccd389 req-dbab1f76-44cf-4d1a-a6cf-327c3b0f9a9e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0622096 May 31 11:57:05.294027 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a07b21b1-91e0-4383-a162-d10a0074edc2 req-783743ea-a529-413b-9c48-baa67d17ef35 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:05.304830 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a07b21b1-91e0-4383-a162-d10a0074edc2 req-783743ea-a529-413b-9c48-baa67d17ef35 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1444893 May 31 11:57:05.416652 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a07b21b1-91e0-4383-a162-d10a0074edc2 req-e84d579b-2c33-4943-a22f-a4eee04cbc32 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1105528 May 31 11:57:05.548089 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-9017883d-1d1a-401a-82f1-6f80e723c36b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:05.549608 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-9017883d-1d1a-401a-82f1-6f80e723c36b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 815 time: 0.2117887 May 31 11:57:05.570072 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-13550a71-cafd-41d5-a3b0-d5b3dabd5585 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/quotas/7dea63b6c4a340d7b361fe91ca6b8281 HTTP/1.1" status: 200 len: 375 time: 0.0133190 May 31 11:57:05.674386 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-dd5cfc20-43e2-4490-98db-1eed91ba1bdc tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0983214 May 31 11:57:05.855982 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-a4650311-6f86-4df6-8e7d-45b0d00bdc61 service neutron] 10.222.0.48 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1743383 May 31 11:57:06.014760 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-e2167c4e-2142-4942-b029-639308d1429a service neutron] 10.222.0.48 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1526623 May 31 11:57:06.434674 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d7015cb-0fc3-4ec0-a12c-71ab93edf8ec req-a4c71a0f-f536-4cae-9b6d-7cd719f4c394 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:06.435241 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-66872957-87e1-44bf-9671-80a94a24c0f2 req-fcdd5443-37eb-4067-ba5b-1c7bee0da046 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 210 time: 0.0607235 May 31 11:57:06.435617 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5d7015cb-0fc3-4ec0-a12c-71ab93edf8ec req-a4c71a0f-f536-4cae-9b6d-7cd719f4c394 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1278105 May 31 11:57:06.519687 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5d7015cb-0fc3-4ec0-a12c-71ab93edf8ec req-4fd1da10-eca9-40ad-9f11-06484a726a4b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0855021 May 31 11:57:06.742341 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-189234e1-c4bf-418c-8f63-09ea15d3a7ac req-9ee85e80-8ca5-4bd5-950d-d7a86edb6b8a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:06.744853 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-189234e1-c4bf-418c-8f63-09ea15d3a7ac req-9ee85e80-8ca5-4bd5-950d-d7a86edb6b8a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0957093 May 31 11:57:06.826354 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-189234e1-c4bf-418c-8f63-09ea15d3a7ac req-7a1151bb-c439-4c97-a798-c79297bfee17 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0740283 May 31 11:57:07.475692 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:57:07.537347 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0632cbd8-6941-4124-93d3-ba23d8899f18 req-1a6e7494-9b90-486f-871f-9a4df566ffaa tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 210 time: 0.0548661 May 31 11:57:07.809979 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8c2a5749-bbdf-4b8f-91a6-f53470c53f70 req-10f47d51-2c63-4295-8f67-5f0e870b6a94 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:07.811592 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8c2a5749-bbdf-4b8f-91a6-f53470c53f70 req-10f47d51-2c63-4295-8f67-5f0e870b6a94 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1200221 May 31 11:57:07.922793 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8c2a5749-bbdf-4b8f-91a6-f53470c53f70 req-ee23b394-d055-444a-8a0e-6ebee0839829 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1035883 May 31 11:57:08.087202 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d592e5aa-4c00-4d16-811f-1341a42c4785 req-8bcc26ee-d818-44fc-98fb-cb8d7268691c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:08.094117 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d592e5aa-4c00-4d16-811f-1341a42c4785 req-8bcc26ee-d818-44fc-98fb-cb8d7268691c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0942180 May 31 11:57:08.151565 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d592e5aa-4c00-4d16-811f-1341a42c4785 req-ac59d594-95a5-4d51-8536-178cd9453921 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0565441 May 31 11:57:08.758782 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3f66827b-5617-4ced-acde-fb7b29c574b9 req-ff3eee7f-e88e-45fb-b310-0ce5f7d5a4d2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 210 time: 0.0954201 May 31 11:57:09.079804 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:09.195438 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bde27d87-8c41-4f42-9665-061e21f9f9c4 req-4181b30c-0440-43bd-b71f-6dd94227c0d8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:09.203794 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bde27d87-8c41-4f42-9665-061e21f9f9c4 req-4181b30c-0440-43bd-b71f-6dd94227c0d8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1030419 May 31 11:57:09.248562 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bde27d87-8c41-4f42-9665-061e21f9f9c4 req-53ab206f-3175-4c6e-b217-587e838c4680 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0436039 May 31 11:57:09.491058 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f5c03ca2-403b-4dad-810a-3facb47d3b02 req-08de21e1-935b-4ca6-b2bd-244846696d59 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:09.499935 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f5c03ca2-403b-4dad-810a-3facb47d3b02 req-08de21e1-935b-4ca6-b2bd-244846696d59 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1238544 May 31 11:57:09.575753 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f5c03ca2-403b-4dad-810a-3facb47d3b02 req-422f646e-10e4-4f1b-a84c-de802f837b51 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0743356 May 31 11:57:09.972251 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0f440ec8-9b03-4ed6-8777-381e8f384650 req-576b9c67-ee59-4ade-a8a0-bbc1e41c950b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 210 time: 0.0622513 May 31 11:57:10.197505 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33260) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:10.383945 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-19192d32-bfa6-4054-95f8-9b4c683fc51f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:10.385676 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-19192d32-bfa6-4054-95f8-9b4c683fc51f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 815 time: 0.2492077 May 31 11:57:10.444179 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Request body: {'port': {'device_id': '1c76757c-f367-4ba4-8e44-150f317f153e', 'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'admin_state_up': True, 'tenant_id': '7dea63b6c4a340d7b361fe91ca6b8281'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:10.660025 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:10.689471 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:10.694420 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Usage tracker for resource:security_group_rule and tenant:09ef9b1eb85e4612830a5d5293d9a770 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:10.710829 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:10.743913 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:10.773924 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a69235ed-b397-4c58-b261-9113a54ea2ef req-7b2e1d04-4bcf-4664-b087-5326c52b2283 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:10.775666 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a69235ed-b397-4c58-b261-9113a54ea2ef req-7b2e1d04-4bcf-4664-b087-5326c52b2283 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.3726392 May 31 11:57:10.839645 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a69235ed-b397-4c58-b261-9113a54ea2ef req-dd4c7757-d12a-4f89-bafd-0a962368802d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0462875 May 31 11:57:10.884255 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:10.899338 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:10.912078 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbcd75fe-ff97-4b73-9a87-de37b0880604 req-733cb3c2-6598-4bab-a671-d858f16679c5 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:10.913642 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbcd75fe-ff97-4b73-9a87-de37b0880604 req-733cb3c2-6598-4bab-a671-d858f16679c5 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1743517 May 31 11:57:10.975400 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:10.989126 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbcd75fe-ff97-4b73-9a87-de37b0880604 req-a48cb4c8-51a3-4a0b-9c63-dc4b0fbd3046 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0630190 May 31 11:57:10.993901 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Made reservation on behalf of 7dea63b6c4a340d7b361fe91ca6b8281 for: {'port': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:57:10.997965 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:11.065064 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:11.075045 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-59d1039e-8f41-4abc-9a1e-800f590d0387 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:11.204874 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-09cb0249-8006-4ab5-bdf4-ff584c90b5ff req-00c882f8-cfad-4c7b-8e03-b75796e54a91 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 210 time: 0.0666180 May 31 11:57:11.207454 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=09ef9b1eb85e4612830a5d5293d9a770&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0082369 May 31 11:57:11.216542 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=fc95def7-f95b-4f7b-bfa9-02c36608b57d,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:11.218496 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33302) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:11.219474 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:11.315985 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Allocated IP 172.20.0.11 (709626b6-807b-4256-9031-1578948f1fe8/4c58bc8a-d411-461e-a483-39773ff0d6f4/24ab9cdc-1297-4bcd-aea2-a477b3dd537c) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:57:11.338502 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-03180dbb-5e01-438c-b295-739aeba7c9a2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:11.370643 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-03180dbb-5e01-438c-b295-739aeba7c9a2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:11.521273 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-03180dbb-5e01-438c-b295-739aeba7c9a2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:11.529355 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c5dcfcd8-1fcd-4158-8648-ddac36f3a2c4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:11.548321 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03180dbb-5e01-438c-b295-739aeba7c9a2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=fc95def7-f95b-4f7b-bfa9-02c36608b57d,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:11.551383 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-03180dbb-5e01-438c-b295-739aeba7c9a2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:11.560281 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-03180dbb-5e01-438c-b295-739aeba7c9a2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:11.561774 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-03180dbb-5e01-438c-b295-739aeba7c9a2 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/fc95def7-f95b-4f7b-bfa9-02c36608b57d HTTP/1.1" status: 204 len: 168 time: 0.3416114 May 31 11:57:11.613245 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:11.719822 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:11.728445 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:11.732849 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['c4a233f2-9446-42a7-964d-1dc2496c938a', '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8', '150f3a3b-28b5-435a-b826-0eee61f6b143'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:57:11.813283 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Pushing event updated for resources: {'Port': ['ID=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:11.825501 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:11.838287 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Network 709626b6-807b-4256-9031-1578948f1fe8 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:57:11.839691 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:57:11.842516 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:57:11.963499 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:11.964337 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:12.175573 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.7705986 May 31 11:57:12.187584 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Request body: {'port': {'device_id': '1c76757c-f367-4ba4-8e44-150f317f153e', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:12.302929 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f52e85a8-8da2-4369-8ead-d366ad0c36b8 req-c6e549be-0fa6-4fa8-864f-27fd8f9c5eec tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:12.310591 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f52e85a8-8da2-4369-8ead-d366ad0c36b8 req-c6e549be-0fa6-4fa8-864f-27fd8f9c5eec tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1822634 May 31 11:57:12.366943 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f52e85a8-8da2-4369-8ead-d366ad0c36b8 req-b71f62a1-d5b1-430e-8571-f9489e63e8a9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0553482 May 31 11:57:12.433916 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:12.458751 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3898becd-bac3-499b-9cef-a9491b48c207 req-153b0235-88c0-491f-9d0e-d1f49ee8e44b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:12.465972 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3898becd-bac3-499b-9cef-a9491b48c207 req-153b0235-88c0-491f-9d0e-d1f49ee8e44b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.2338593 May 31 11:57:12.573947 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3898becd-bac3-499b-9cef-a9491b48c207 req-b7ede7ce-b2da-4bd2-923e-be84835756cc tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1070940 May 31 11:57:12.630250 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8396e768-5dd0-4b41-96c0-c16669b9b4dc req-8c97ceec-4499-491c-b3a0-3caf7d4f5078 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:12.631738 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8396e768-5dd0-4b41-96c0-c16669b9b4dc req-8c97ceec-4499-491c-b3a0-3caf7d4f5078 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 932 time: 0.1216691 May 31 11:57:12.682183 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8396e768-5dd0-4b41-96c0-c16669b9b4dc req-9375f03e-638f-4899-806a-f64f08f4dddb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0446417 May 31 11:57:12.861259 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 24ab9cdc-1297-4bcd-aea2-a477b3dd537c {{(pid=99039) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:57:12.872284 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:12.892254 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Transition to ACTIVE for port object 24ab9cdc-1297-4bcd-aea2-a477b3dd537c will not be triggered until provisioned by entity L2. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:57:12.965324 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:12.969775 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8a0484ee-26ce-47d9-b964-5eb897cb891e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:13.041938 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33364) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:13.158540 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:13.183716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:13.199661 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Usage tracker for resource:security_group_rule and tenant:f22caf4967014e28a913779cc5fe0467 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:13.203863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:13.211268 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:13.226599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Pushing event updated for resources: {'Port': ['ID=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:13.230191 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.261s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:13.345449 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:13.462885 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:13.471204 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3ab9f170-1588-47e2-bcc2-53a7dc05f2e3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:13.476124 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Attempting to bind port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:57:13.476635 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Attempting to bind port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:57:13.476726 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Attempting to bind port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c on network 709626b6-807b-4256-9031-1578948f1fe8 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:57:13.481967 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c on network 709626b6-807b-4256-9031-1578948f1fe8 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:57:13.482371 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Attempting to bind port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c on network 709626b6-807b-4256-9031-1578948f1fe8 {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:57:13.487895 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 57, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:57:13.488415 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Checking segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:57:13.488808 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Bound using segment: {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'} {{(pid=99039) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:57:13.489298 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.managers [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Bound port: 24ab9cdc-1297-4bcd-aea2-a477b3dd537c, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'a846ec5a-b6ef-4a36-83ff-119eb454231c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 356, 'network_id': '709626b6-807b-4256-9031-1578948f1fe8'}}] {{(pid=99039) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:57:13.490188 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:13.553222 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=eafe20ca-0f53-4abd-9caa-a00fb47ca480,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:13.577582 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:13.643854 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-d9ec1e04-0f53-465c-a88d-99453a3d6fe3 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=f22caf4967014e28a913779cc5fe0467&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6005375 May 31 11:57:13.644673 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33382) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:13.684548 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] For port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c, host hv-cinder-79123, cleared binding levels {{(pid=99039) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:57:13.688491 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] For port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99039) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:57:13.704606 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a8e9352-3e49-44b0-9d97-250c3de79c51 req-40407916-cc86-4264-8211-082a492266a5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:13.707046 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4a8e9352-3e49-44b0-9d97-250c3de79c51 req-40407916-cc86-4264-8211-082a492266a5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1469536 May 31 11:57:13.769395 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:13.775421 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-b2823df6-3615-46cf-ba6a-278860690f86 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:13.816353 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4a8e9352-3e49-44b0-9d97-250c3de79c51 req-24dca762-d76e-4498-9e42-0e7731f79aa5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1031542 May 31 11:57:13.835149 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-afb35011-fee2-4b80-b760-5b125f051e10 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:13.845379 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:13.846570 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:13.849234 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] 10.222.0.47 "PUT /v2.0/ports/24ab9cdc-1297-4bcd-aea2-a477b3dd537c HTTP/1.1" status: 200 len: 1094 time: 1.6654012 May 31 11:57:13.849950 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '1c76757c-f367-4ba4-8e44-150f317f153e', 'tag': '24ab9cdc-1297-4bcd-aea2-a477b3dd537c'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:57:13.853544 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-b98a9bb6-8420-4592-b96b-19198970462a" -d '{"events": [{"name": "network-changed", "server_uuid": "1c76757c-f367-4ba4-8e44-150f317f153e", "tag": "24ab9cdc-1297-4bcd-aea2-a477b3dd537c"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:57:13.882821 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Pushing event updated for resources: {'Port': ['ID=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,revision_number=3']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:13.887850 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:13.897192 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-afb35011-fee2-4b80-b760-5b125f051e10 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:13.962249 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-313a5702-865c-4ea0-9046-e335adb13379 req-2aaf1f59-8963-4221-8ce7-9b09d8aa3dee tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:13.965020 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-313a5702-865c-4ea0-9046-e335adb13379 req-2aaf1f59-8963-4221-8ce7-9b09d8aa3dee tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.2031105 May 31 11:57:14.032251 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-313a5702-865c-4ea0-9046-e335adb13379 req-8febe6c1-62fc-4798-8c29-9f47b1f2fe92 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0520754 May 31 11:57:14.154756 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-a4be3c35-43c3-4e40-8389-dae5b3383766 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:14.157597 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-a4be3c35-43c3-4e40-8389-dae5b3383766 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 1097 time: 0.1867313 May 31 11:57:14.167066 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34585dfc-8ac3-47a7-8b15-a6d777edc61a req-a6b92e80-6fa4-4aea-8542-009dcc3e3f25 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:14.167762 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-34585dfc-8ac3-47a7-8b15-a6d777edc61a req-a6b92e80-6fa4-4aea-8542-009dcc3e3f25 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:57:14.220657 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-afb35011-fee2-4b80-b760-5b125f051e10 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:14.234077 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ede3807a-38af-4008-a5af-cb6c014acccd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:14.243855 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-ff806973-5dba-420a-b484-91a94126cf73 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=24ab9cdc-1297-4bcd-aea2-a477b3dd537c HTTP/1.1" status: 200 len: 217 time: 0.0730748 May 31 11:57:14.273588 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afb35011-fee2-4b80-b760-5b125f051e10 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=eafe20ca-0f53-4abd-9caa-a00fb47ca480,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:14.298706 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-afb35011-fee2-4b80-b760-5b125f051e10 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.065s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:14.300162 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-afb35011-fee2-4b80-b760-5b125f051e10 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:14.307798 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-afb35011-fee2-4b80-b760-5b125f051e10 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/eafe20ca-0f53-4abd-9caa-a00fb47ca480 HTTP/1.1" status: 204 len: 168 time: 0.6614692 May 31 11:57:14.345107 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-94017d4d-1dad-4ebc-883e-9dadf80e75d1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:14.346852 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-94017d4d-1dad-4ebc-883e-9dadf80e75d1 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 200 len: 855 time: 0.0926843 May 31 11:57:14.416212 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-e57fc30d-7b3b-4aa9-854e-9d63328615dc service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=709626b6-807b-4256-9031-1578948f1fe8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0608337 May 31 11:57:14.463874 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-34585dfc-8ac3-47a7-8b15-a6d777edc61a req-a6b92e80-6fa4-4aea-8542-009dcc3e3f25 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:57:14.470140 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-34585dfc-8ac3-47a7-8b15-a6d777edc61a req-a6b92e80-6fa4-4aea-8542-009dcc3e3f25 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 944 time: 0.4313636 May 31 11:57:14.637868 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-34585dfc-8ac3-47a7-8b15-a6d777edc61a req-f42170a6-e281-4870-8e3f-04451a15fe9c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1618271 May 31 11:57:14.682239 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-08dad749-ed4e-4b16-b259-433a1d8299e9 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2567866 May 31 11:57:14.898411 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-acc668cd-3005-4b27-8ac8-afa63f1bf50c req-1a494592-0e45-481d-a949-d4a6aaf92bb7 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2028918 May 31 11:57:14.900078 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:57:13 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-373a40b4-44ed-41bb-a1ed-799a522b9490 x-openstack-request-id: req-373a40b4-44ed-41bb-a1ed-799a522b9490 {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:57:14.900498 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "1c76757c-f367-4ba4-8e44-150f317f153e", "tag": "24ab9cdc-1297-4bcd-aea2-a477b3dd537c", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:57:14.900978 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-373a40b4-44ed-41bb-a1ed-799a522b9490 {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:57:14.901504 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '1c76757c-f367-4ba4-8e44-150f317f153e', 'tag': '24ab9cdc-1297-4bcd-aea2-a477b3dd537c', 'status': 'completed', 'code': 200} May 31 11:57:14.987882 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33410) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:15.246645 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b98a9bb6-8420-4592-b96b-19198970462a req-6b2d90ac-2c7e-43ff-85ea-bbf209c3030b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.248984 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b98a9bb6-8420-4592-b96b-19198970462a req-6b2d90ac-2c7e-43ff-85ea-bbf209c3030b service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 1097 time: 0.1542971 May 31 11:57:15.258877 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-742a3b38-ded2-41a5-8596-b80710cba016 req-a4a18393-1c16-4047-92cf-f00d9f73fce1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.266232 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-742a3b38-ded2-41a5-8596-b80710cba016 req-a4a18393-1c16-4047-92cf-f00d9f73fce1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.2372677 May 31 11:57:15.307385 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:15.340195 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:15.347781 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Usage tracker for resource:security_group_rule and tenant:60978a777bec43adbbdb70e6454c2524 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:15.361868 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:15.378063 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:15.408749 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-742a3b38-ded2-41a5-8596-b80710cba016 req-1ae7abf8-b143-469d-9203-d059b4b0dfbc tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1413002 May 31 11:57:15.450024 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33420) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:15.455340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-463b05e9-54e5-4593-9cc6-ed3f57b46095 req-11cc25b1-5d8d-4af2-a287-44cd611f02b8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.456904 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-463b05e9-54e5-4593-9cc6-ed3f57b46095 req-11cc25b1-5d8d-4af2-a287-44cd611f02b8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.3521383 May 31 11:57:15.469444 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db0efa58-3723-431a-8499-25cc1300d03c req-f423a488-a146-4238-931e-dbe147d543ae service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.469444 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0efa58-3723-431a-8499-25cc1300d03c req-f423a488-a146-4238-931e-dbe147d543ae service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1099 time: 0.2447085 May 31 11:57:15.481064 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:15.511730 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:15.520469 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Usage tracker for resource:security_group_rule and tenant:6f5c97149b6f4f1086f218a9906f3c1b is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:15.525602 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:15.529057 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b98a9bb6-8420-4592-b96b-19198970462a req-1c62b1f4-5413-453e-8ff9-c241a209a450 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.530717 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b98a9bb6-8420-4592-b96b-19198970462a req-1c62b1f4-5413-453e-8ff9-c241a209a450 service neutron] 10.222.0.47 "GET /v2.0/networks?id=709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 200 len: 916 time: 0.2614555 May 31 11:57:15.534768 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:15.678997 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b98a9bb6-8420-4592-b96b-19198970462a req-54bd34b5-be73-4969-94ab-f9fdc3d62647 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=24ab9cdc-1297-4bcd-aea2-a477b3dd537c HTTP/1.1" status: 200 len: 217 time: 0.1354344 May 31 11:57:15.697102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db0efa58-3723-431a-8499-25cc1300d03c req-72219a47-c446-477a-bbe6-3b6e0062dba2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.698223 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0efa58-3723-431a-8499-25cc1300d03c req-72219a47-c446-477a-bbe6-3b6e0062dba2 service neutron] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 920 time: 0.1739833 May 31 11:57:15.704466 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-463b05e9-54e5-4593-9cc6-ed3f57b46095 req-63cc1394-c0d4-47fe-a99a-bbf7f519b250 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2204890 May 31 11:57:15.799857 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b98a9bb6-8420-4592-b96b-19198970462a req-3e43c3cd-f37a-4863-8b5e-187985871fa8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.801471 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b98a9bb6-8420-4592-b96b-19198970462a req-3e43c3cd-f37a-4863-8b5e-187985871fa8 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 200 len: 855 time: 0.1014769 May 31 11:57:15.819430 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0efa58-3723-431a-8499-25cc1300d03c req-fb14665f-ddcf-4f87-8857-c03963091d68 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778 HTTP/1.1" status: 200 len: 217 time: 0.1072674 May 31 11:57:15.864190 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b98a9bb6-8420-4592-b96b-19198970462a req-6202d9a5-3cf1-404e-9834-757d4620f256 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=709626b6-807b-4256-9031-1578948f1fe8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0551419 May 31 11:57:15.876032 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-84cdd6cc-ffb9-4661-bb49-44ab98b0f5a4 req-6ebeb35e-1617-41eb-bc50-e9bbee63d202 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.883935 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-84cdd6cc-ffb9-4661-bb49-44ab98b0f5a4 req-6ebeb35e-1617-41eb-bc50-e9bbee63d202 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.2586262 May 31 11:57:15.957223 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:15.982374 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db0efa58-3723-431a-8499-25cc1300d03c req-1d80b097-4fe1-4ab9-beef-94b27bc9d70a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:15.983825 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0efa58-3723-431a-8499-25cc1300d03c req-1d80b097-4fe1-4ab9-beef-94b27bc9d70a service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.1452832 May 31 11:57:16.034940 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.048817 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ecdc24d6-8029-4b89-bc27-868a791d0d9d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:16.060295 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0efa58-3723-431a-8499-25cc1300d03c req-1ea6a7d7-6768-45c1-874f-b48c1f9d50f8 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0656743 May 31 11:57:16.127411 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=60978a777bec43adbbdb70e6454c2524&name=default HTTP/1.1" status: 200 len: 2979 time: 1.1372154 May 31 11:57:16.138463 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-732b08d3-0ab2-4d5d-b801-ed1d8ea1ff76 req-8867ae3b-2149-4aee-a2da-102498408161 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:16.145922 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-732b08d3-0ab2-4d5d-b801-ed1d8ea1ff76 req-8867ae3b-2149-4aee-a2da-102498408161 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 944 time: 0.2652822 May 31 11:57:16.147865 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=acc7c38b-f9ba-410d-9f45-826a5475c73f,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:16.157254 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33428) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:16.191216 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-84cdd6cc-ffb9-4661-bb49-44ab98b0f5a4 req-61ee8305-c22f-474a-9e60-322a489b8389 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.3062274 May 31 11:57:16.200920 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.224816 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:16.296248 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.302328 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-732b08d3-0ab2-4d5d-b801-ed1d8ea1ff76 req-5ab8eaff-e2f6-4b94-8114-6734f32fe49d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1551690 May 31 11:57:16.306089 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b98a9bb6-8420-4592-b96b-19198970462a req-b7e0369c-19ce-46f7-92d0-d21b2254253c service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4191344 May 31 11:57:16.308746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-84488dbd-25a2-45bf-945f-b49a93d6a1fc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:16.315765 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5d47551d-f708-4c2e-b565-bb1a69787d5b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.335217 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0efa58-3723-431a-8499-25cc1300d03c req-ced9e176-57a1-44f8-a948-81b3e3e2f4d2 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2585313 May 31 11:57:16.457999 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=6f5c97149b6f4f1086f218a9906f3c1b&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0068548 May 31 11:57:16.467343 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5d47551d-f708-4c2e-b565-bb1a69787d5b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.476804 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33434) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:16.477694 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=552847f0-5460-407c-b904-cf521d3693d8,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:16.480664 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.172s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:16.539231 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0efa58-3723-431a-8499-25cc1300d03c req-23a7be31-6950-4a8e-9a3e-624b9b7c8603 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1956496 May 31 11:57:16.677812 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b98a9bb6-8420-4592-b96b-19198970462a req-bed04298-c0b0-4105-b4cb-f86b48b21684 service neutron] 10.222.0.47 "GET /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3564351 May 31 11:57:16.722576 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-49263a23-98a9-40c0-922c-c4da1ab9e9ea admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.764643 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-24b656d9-de33-432b-81a5-43a4acda0116 req-fc96d100-6ad1-438b-8088-e3e7897779e4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:16.772883 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-24b656d9-de33-432b-81a5-43a4acda0116 req-fc96d100-6ad1-438b-8088-e3e7897779e4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.2074294 May 31 11:57:16.793093 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-49263a23-98a9-40c0-922c-c4da1ab9e9ea admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.841250 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5d47551d-f708-4c2e-b565-bb1a69787d5b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.868989 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d10c289c-2a1b-40a5-aaf4-26959cb9a400 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:16.887053 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-24b656d9-de33-432b-81a5-43a4acda0116 req-3aaaebb1-6a9b-4fc8-9d6a-be9356f50acf tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1003635 May 31 11:57:16.889932 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d47551d-f708-4c2e-b565-bb1a69787d5b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=acc7c38b-f9ba-410d-9f45-826a5475c73f,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:16.902962 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5d47551d-f708-4c2e-b565-bb1a69787d5b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:16.908970 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5d47551d-f708-4c2e-b565-bb1a69787d5b admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:16.914441 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-5d47551d-f708-4c2e-b565-bb1a69787d5b admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/acc7c38b-f9ba-410d-9f45-826a5475c73f HTTP/1.1" status: 204 len: 168 time: 0.7539365 May 31 11:57:16.969623 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b685f0c-7e56-4f20-9c40-eb0c9bb827b0 req-7d0d6046-4b13-4e5b-9d0c-529cde03baa8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:16.969623 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5b685f0c-7e56-4f20-9c40-eb0c9bb827b0 req-7d0d6046-4b13-4e5b-9d0c-529cde03baa8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1374576 May 31 11:57:17.031887 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5b685f0c-7e56-4f20-9c40-eb0c9bb827b0 req-c1ffef04-4f10-49cd-a7b9-65267008b055 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0557132 May 31 11:57:17.039449 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-49263a23-98a9-40c0-922c-c4da1ab9e9ea admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:17.043396 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-80da2f7d-642f-49fa-b014-1d26e540604c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:17.065790 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49263a23-98a9-40c0-922c-c4da1ab9e9ea admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=552847f0-5460-407c-b904-cf521d3693d8,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:17.071362 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-49263a23-98a9-40c0-922c-c4da1ab9e9ea admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:17.075223 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-49263a23-98a9-40c0-922c-c4da1ab9e9ea admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:17.078860 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-49263a23-98a9-40c0-922c-c4da1ab9e9ea admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/552847f0-5460-407c-b904-cf521d3693d8 HTTP/1.1" status: 204 len: 168 time: 0.5943990 May 31 11:57:17.402156 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-264e8dd0-519e-416a-a743-34c456e1809c req-f7cafc1d-3161-44d5-abb8-36d264b2f18b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:17.403638 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-264e8dd0-519e-416a-a743-34c456e1809c req-f7cafc1d-3161-44d5-abb8-36d264b2f18b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0835624 May 31 11:57:17.450572 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-264e8dd0-519e-416a-a743-34c456e1809c req-3c041847-2454-4601-a6d0-7350c4aaf6a1 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0410905 May 31 11:57:17.483209 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95f4f9c6-9f44-431b-a3cf-591f0968d088 req-60fe0e8b-ddc7-4ed7-8823-7e2f1397c9c8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:17.484664 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-95f4f9c6-9f44-431b-a3cf-591f0968d088 req-60fe0e8b-ddc7-4ed7-8823-7e2f1397c9c8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 944 time: 0.0797896 May 31 11:57:17.545191 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-95f4f9c6-9f44-431b-a3cf-591f0968d088 req-1574deaa-23c8-491a-be09-136ddf2e1e13 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0539689 May 31 11:57:18.242486 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24ab9cdc-1297-4bcd-aea2-a477b3dd537c details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:57:18.263209 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d288f15a-80b3-456e-a49b-6c2a8c2dbeb6 req-d0b6fb12-a571-418c-aafe-bddc34d7a0d7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:18.274658 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d288f15a-80b3-456e-a49b-6c2a8c2dbeb6 req-d0b6fb12-a571-418c-aafe-bddc34d7a0d7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1971066 May 31 11:57:18.419209 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d288f15a-80b3-456e-a49b-6c2a8c2dbeb6 req-7b7312e1-8f72-49ae-af0b-562acbe8c74a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1371138 May 31 11:57:18.468692 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e17dce2a-414b-4460-9cd8-069a36d09d6c req-e5f71678-a3c3-4b51-8fcc-4ae8281d88f2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:18.471474 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e17dce2a-414b-4460-9cd8-069a36d09d6c req-e5f71678-a3c3-4b51-8fcc-4ae8281d88f2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1718185 May 31 11:57:18.518932 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:18.531837 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '24ab9cdc-1297-4bcd-aea2-a477b3dd537c', 'network_id': '709626b6-807b-4256-9031-1578948f1fe8', 'port_id': '24ab9cdc-1297-4bcd-aea2-a477b3dd537c', 'mac_address': 'fa:16:3e:eb:fc:8c', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 356, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c58bc8a-d411-461e-a483-39773ff0d6f4', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:57:18.590367 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e17dce2a-414b-4460-9cd8-069a36d09d6c req-22d9a3fb-9cf5-4cd7-9935-f96d729f5720 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1043265 May 31 11:57:18.606023 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:18.624364 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 24ab9cdc-1297-4bcd-aea2-a477b3dd537c {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:57:18.641737 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:18.686578 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33480) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:18.714571 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:18.726094 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:18.728506 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:18.801087 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-489c6450-37fb-4b63-9a13-950645e026be req-00c23218-53a8-40a4-9d8f-7d44fac100b6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:18.802997 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-489c6450-37fb-4b63-9a13-950645e026be req-00c23218-53a8-40a4-9d8f-7d44fac100b6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1545391 May 31 11:57:18.815198 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:18.819350 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:18.875485 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-489c6450-37fb-4b63-9a13-950645e026be req-f299d2a8-768d-4951-9a31-3c0301a78d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0665929 May 31 11:57:18.911867 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:18.920355 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Usage tracker for resource:security_group_rule and tenant:1d2212bfec5348d0963a30297bb7aaab is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:18.924411 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:18.935460 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:18.994204 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2cc6cb81-ef7e-4363-93ea-c63b080ef4c0 req-018f7660-9445-4abb-a72e-584c47c3ca9a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:18.995684 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2cc6cb81-ef7e-4363-93ea-c63b080ef4c0 req-018f7660-9445-4abb-a72e-584c47c3ca9a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 945 time: 0.2505188 May 31 11:57:19.075387 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2cc6cb81-ef7e-4363-93ea-c63b080ef4c0 req-7c73e295-591d-4e60-9d20-367e605baf35 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0729358 May 31 11:57:19.084574 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:19.132233 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['24ab9cdc-1297-4bcd-aea2-a477b3dd537c'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:57:19.187490 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:19.232651 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bbe81f1f-27ce-446d-8eb5-03444478488f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:19.241118 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=1d2212bfec5348d0963a30297bb7aaab&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5534804 May 31 11:57:19.242925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e06bf2ea-ccdb-4dc1-a87d-b37cd62dd8c0,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:19.244039 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33496) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:19.251086 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.251442 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.251791 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.252164 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.252479 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.252809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.253136 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.253474 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.254009 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.254746 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.254746 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.255265 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:19.261408 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:19.306901 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-4c343fec-626a-4a5f-b136-2eb0ad4a078c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:19.333952 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-4c343fec-626a-4a5f-b136-2eb0ad4a078c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:19.451375 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-4c343fec-626a-4a5f-b136-2eb0ad4a078c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:19.456719 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-9f177693-20d7-499e-b996-1f5eb9c10c4b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:19.469894 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4c343fec-626a-4a5f-b136-2eb0ad4a078c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e06bf2ea-ccdb-4dc1-a87d-b37cd62dd8c0,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:19.472139 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-4c343fec-626a-4a5f-b136-2eb0ad4a078c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:19.477657 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-4c343fec-626a-4a5f-b136-2eb0ad4a078c admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:19.493154 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-4c343fec-626a-4a5f-b136-2eb0ad4a078c admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/e06bf2ea-ccdb-4dc1-a87d-b37cd62dd8c0 HTTP/1.1" status: 204 len: 168 time: 0.2477012 May 31 11:57:19.681412 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bae2a2bb-fa7c-49e2-a857-4f1ee24f587e req-a9d4e387-ac82-49d2-b2c4-b730694bb377 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:19.682827 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bae2a2bb-fa7c-49e2-a857-4f1ee24f587e req-a9d4e387-ac82-49d2-b2c4-b730694bb377 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.0981412 May 31 11:57:19.729744 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bae2a2bb-fa7c-49e2-a857-4f1ee24f587e req-7c4169b3-4d31-4e74-9fcc-d8934678c98a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409219 May 31 11:57:19.806282 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d94b7280-1f68-49d3-a42f-4895724c29ad req-1e2c3e61-202c-4985-86c3-805390923ac9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:19.807739 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d94b7280-1f68-49d3-a42f-4895724c29ad req-1e2c3e61-202c-4985-86c3-805390923ac9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0874274 May 31 11:57:19.853286 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d94b7280-1f68-49d3-a42f-4895724c29ad req-d450d7a9-123a-4479-a579-92cc6df2eb92 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397923 May 31 11:57:20.079164 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3292c5d7-6cc2-486d-84fe-eb518d190ab1 req-0a677083-5265-4165-90f8-a20e62c6b149 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:20.080718 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3292c5d7-6cc2-486d-84fe-eb518d190ab1 req-0a677083-5265-4165-90f8-a20e62c6b149 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0849380 May 31 11:57:20.127482 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3292c5d7-6cc2-486d-84fe-eb518d190ab1 req-6909d997-2954-47be-b73c-5247d799257e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0406570 May 31 11:57:20.277542 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8176656c-f4fc-4094-acab-4f1a83c8e752 req-1935a6b3-9b03-4c4f-bc6f-5c437a18df46 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:20.279153 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8176656c-f4fc-4094-acab-4f1a83c8e752 req-1935a6b3-9b03-4c4f-bc6f-5c437a18df46 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 945 time: 0.0932457 May 31 11:57:20.326702 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8176656c-f4fc-4094-acab-4f1a83c8e752 req-0362456c-c1a8-4b43-a9af-588c212a4896 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416639 May 31 11:57:20.584315 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-0bf565f9-f4cb-47a2-b31c-7553ead77c23 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:20.585821 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-0bf565f9-f4cb-47a2-b31c-7553ead77c23 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 1098 time: 0.0907738 May 31 11:57:20.760725 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-61e86226-cd2c-4ffa-9ae2-982ecca3b9d2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:20.764155 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-61e86226-cd2c-4ffa-9ae2-982ecca3b9d2 service neutron] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 917 time: 0.1645606 May 31 11:57:20.876709 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-5ca70d4c-64dc-4205-979f-1374e5a13b44 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98 HTTP/1.1" status: 200 len: 1001 time: 0.1026416 May 31 11:57:20.890660 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5a357b65-5d34-423a-8311-ce663ba774e7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:20.998920 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-e90c1913-929d-4281-aac7-e010c54433e8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:21.000326 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-e90c1913-929d-4281-aac7-e010c54433e8 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0917194 May 31 11:57:21.049692 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6bc59ec0-4c6e-408d-995f-1247ed12fac6 req-b99317a4-6d10-4697-9b97-3cd9fb6a0def tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:21.051919 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6bc59ec0-4c6e-408d-995f-1247ed12fac6 req-b99317a4-6d10-4697-9b97-3cd9fb6a0def tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0839765 May 31 11:57:21.103823 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6bc59ec0-4c6e-408d-995f-1247ed12fac6 req-0e52e20f-9a10-47b0-87e7-73eab14afec9 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0445185 May 31 11:57:21.162163 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c78f8236-0a33-49d6-8e08-72313c51c121 req-f740ba25-92d5-4415-9fcd-3daaeac01558 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:21.169536 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c78f8236-0a33-49d6-8e08-72313c51c121 req-f740ba25-92d5-4415-9fcd-3daaeac01558 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.2862065 May 31 11:57:21.230289 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c78f8236-0a33-49d6-8e08-72313c51c121 req-56083384-dae3-4169-a1f8-1f58ef663405 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0503919 May 31 11:57:21.234966 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-088ccb92-71c7-4bf8-a40e-8d5f7ff48921 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:21.236368 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-088ccb92-71c7-4bf8-a40e-8d5f7ff48921 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2275541 May 31 11:57:21.333438 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62ead9ea-477a-4bac-b925-952f68bafaab req-0d2ffd4b-3d70-491a-aba4-f42eaffa4600 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:21.335086 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-62ead9ea-477a-4bac-b925-952f68bafaab req-0d2ffd4b-3d70-491a-aba4-f42eaffa4600 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0959570 May 31 11:57:21.383370 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-c9e4343f-cda9-4081-b476-303f3fcfafbb service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1385510 May 31 11:57:21.386695 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-62ead9ea-477a-4bac-b925-952f68bafaab req-b8147eeb-21c2-4636-96bc-12003f9054aa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0459609 May 31 11:57:21.577919 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2664e15a-afa5-4575-8478-33a0a4edb871 req-2980db58-5f9a-478f-b7d9-15ccb56f2037 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1876187 May 31 11:57:21.596181 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a5373bb-d9bd-46fc-8a18-7da357f28cac req-4186381d-c74d-4468-8b4a-b86b70afc05b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:21.597700 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9a5373bb-d9bd-46fc-8a18-7da357f28cac req-4186381d-c74d-4468-8b4a-b86b70afc05b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 945 time: 0.1356692 May 31 11:57:21.643957 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9a5373bb-d9bd-46fc-8a18-7da357f28cac req-e7ff22e7-f65a-45f0-9f5b-b98d1d0246ee tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0400562 May 31 11:57:22.143795 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71a6466f-a595-4980-9d57-6aebfdefa499 req-eed71555-59d5-4249-8fbb-0176db467a3d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:22.146728 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-71a6466f-a595-4980-9d57-6aebfdefa499 req-eed71555-59d5-4249-8fbb-0176db467a3d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.0874584 May 31 11:57:22.198195 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-71a6466f-a595-4980-9d57-6aebfdefa499 req-9740e37c-bcf3-4702-9d8a-8381e7848eba tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0439842 May 31 11:57:22.221289 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33592) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:22.270830 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24ab9cdc-1297-4bcd-aea2-a477b3dd537c up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:57:22.307473 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-24ac86e2-2bd6-4b32-9a10-78cbb1ad9b45 req-1dd54b83-cd96-450d-9eba-500e02af7d01 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:22.309442 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-24ac86e2-2bd6-4b32-9a10-78cbb1ad9b45 req-1dd54b83-cd96-450d-9eba-500e02af7d01 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0892167 May 31 11:57:22.386465 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-24ac86e2-2bd6-4b32-9a10-78cbb1ad9b45 req-ef6fb176-2347-4a53-88a3-e12c349f5472 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0692627 May 31 11:57:22.475719 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b203937-bd15-40a1-9980-03483f50ee41 req-53fee941-66ba-4139-abd2-ad1f769dba67 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:22.477514 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8b203937-bd15-40a1-9980-03483f50ee41 req-53fee941-66ba-4139-abd2-ad1f769dba67 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 945 time: 0.1074200 May 31 11:57:22.502952 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:57:22.509414 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:57:22.509889 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:22.609512 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fe7d2eb3-23e0-4b90-b7c1-25f72ade2147 req-3b407d37-2f6c-4841-9cd5-cdf6c913b9a2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:22.616264 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8b203937-bd15-40a1-9980-03483f50ee41 req-d0681008-0832-4746-8c34-b29b5bd0bfb6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1320641 May 31 11:57:22.617816 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fe7d2eb3-23e0-4b90-b7c1-25f72ade2147 req-3b407d37-2f6c-4841-9cd5-cdf6c913b9a2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0992284 May 31 11:57:22.666619 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-15675137-477c-48fe-acba-69a104dbe39a tempest-TestMinimumBasicScenario-663128781 tempest-TestMinimumBasicScenario-663128781-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:22.668365 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-15675137-477c-48fe-acba-69a104dbe39a tempest-TestMinimumBasicScenario-663128781 tempest-TestMinimumBasicScenario-663128781-project-admin] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1093 time: 0.4459629 May 31 11:57:22.673065 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33608) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:22.678511 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'port_id': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'fixed_ip_address': '172.20.0.4'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:22.680195 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fe7d2eb3-23e0-4b90-b7c1-25f72ade2147 req-d49e4341-eb41-4db2-92b4-63202b5e3466 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0613260 May 31 11:57:22.684488 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:22.686367 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:22.697734 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Usage tracker for resource:floatingip and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:22.701369 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:22.708118 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:22.721100 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '1c76757c-f367-4ba4-8e44-150f317f153e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '24ab9cdc-1297-4bcd-aea2-a477b3dd537c'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:57:22.723402 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-ec5c0340-ccf7-456d-8f62-fe1259835c36" -d '{"events": [{"server_uuid": "1c76757c-f367-4ba4-8e44-150f317f153e", "name": "network-vif-plugged", "status": "completed", "tag": "24ab9cdc-1297-4bcd-aea2-a477b3dd537c"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:57:22.731939 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:22.772101 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:57:22 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-355505bc-a590-4b68-9cf5-5be043accefb x-openstack-request-id: req-355505bc-a590-4b68-9cf5-5be043accefb {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:57:22.772101 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "1c76757c-f367-4ba4-8e44-150f317f153e", "name": "network-vif-plugged", "status": "completed", "tag": "24ab9cdc-1297-4bcd-aea2-a477b3dd537c", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:57:22.772101 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-355505bc-a590-4b68-9cf5-5be043accefb {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:57:22.772101 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '1c76757c-f367-4ba4-8e44-150f317f153e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '24ab9cdc-1297-4bcd-aea2-a477b3dd537c', 'code': 200} May 31 11:57:22.831904 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:22.833385 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:57:22.834346 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for floatingip, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:22.868136 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:22.871602 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d5ed44f-e5c0-4ca9-b2ca-d9b44bd57e7b req-bfa6c7b3-9284-4e63-a0df-e7f9fd9440e1 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:22.874163 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8d5ed44f-e5c0-4ca9-b2ca-d9b44bd57e7b req-bfa6c7b3-9284-4e63-a0df-e7f9fd9440e1 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 945 time: 0.1109774 May 31 11:57:22.980063 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:22.984980 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:23.005871 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8d5ed44f-e5c0-4ca9-b2ca-d9b44bd57e7b req-df8b2989-c2e3-471b-91e5-6344a9bed7a0 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0684385 May 31 11:57:23.060541 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:23.070878 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:23.106714 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:23.204931 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Allocated IP 172.27.1.199 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/834ff228-3f9e-460f-96e4-cd4df6eec21b) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:57:23.241794 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-505e7e9d-b915-45b9-9193-b346ea471609 req-23e88e83-ce2f-419b-9166-ba0ccfba95a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:23.243243 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-505e7e9d-b915-45b9-9193-b346ea471609 req-23e88e83-ce2f-419b-9166-ba0ccfba95a9 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 1099 time: 0.0768652 May 31 11:57:23.248436 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:23.250224 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:23.293129 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-505e7e9d-b915-45b9-9193-b346ea471609 req-abdddd7d-824b-42ff-828d-37b1228ded02 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0398231 May 31 11:57:23.329417 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=8']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:23.332491 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:23.361088 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:23.467956 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:23.534817 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:23.539850 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f4d06bba-39e9-4067-8b2e-43482c1bb492 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:23.544194 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-59a0de11-f027-437f-9d3c-f9b8e281838f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:23.589421 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5fd2ef9a-bd93-4295-a954-f833ea963060 req-369b1598-f70f-4783-949b-a764f6e4ae2a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:23.591293 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5fd2ef9a-bd93-4295-a954-f833ea963060 req-369b1598-f70f-4783-949b-a764f6e4ae2a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0897462 May 31 11:57:23.608410 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:57:23.611167 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:57:23.615393 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:57:23.642185 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5fd2ef9a-bd93-4295-a954-f833ea963060 req-0a424a7c-96d1-44ab-a575-702d43686ad8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0440462 May 31 11:57:23.651056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'Port': ['ID=834ff228-3f9e-460f-96e4-cd4df6eec21b,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:23.663571 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:23.895863 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-89d5ed76-74be-47f8-b574-057951d79eb3 req-ed471c79-952e-4fd6-ad53-648bdc47821c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:23.897454 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-89d5ed76-74be-47f8-b574-057951d79eb3 req-ed471c79-952e-4fd6-ad53-648bdc47821c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0904088 May 31 11:57:23.949832 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-89d5ed76-74be-47f8-b574-057951d79eb3 req-c2dcb598-c3a1-4bd3-94be-a32c1110b29a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0453658 May 31 11:57:24.077311 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-054919e8-a909-47d4-b14c-a87b9d01738a req-f29e9dc2-9782-4c0d-9f66-69b217beaed7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:24.078807 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-054919e8-a909-47d4-b14c-a87b9d01738a req-f29e9dc2-9782-4c0d-9f66-69b217beaed7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.3316674 May 31 11:57:24.156727 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-054919e8-a909-47d4-b14c-a87b9d01738a req-e30ad568-1c83-4a5b-bf7c-42ef35d1af3b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0597384 May 31 11:57:24.203550 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:24.253022 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4265334d-06c3-4899-befa-acb7e5da6107 req-09814bf1-232b-4229-9a9e-ff0cc43c6451 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:24.255142 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4265334d-06c3-4899-befa-acb7e5da6107 req-09814bf1-232b-4229-9a9e-ff0cc43c6451 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.1201463 May 31 11:57:24.318356 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4265334d-06c3-4899-befa-acb7e5da6107 req-cffcf25a-4f0a-4de6-98a9-8d955ea6b9f9 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0569818 May 31 11:57:24.356955 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:24.591014 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:24.604687 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:24.690602 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:24.698024 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:24.750336 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:24.771654 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:57:24.774322 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-e1585bf8-e321-4c96-948a-1c7376ded063" -d '{"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-unplugged", "status": "completed", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:57:24.816713 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [207] Connection: close Content-Length: 182 Content-Type: application/json Date: Mon, 31 May 2021 11:57:24 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-021af500-5448-49a1-906a-b92fd1c390e9 x-openstack-request-id: req-021af500-5448-49a1-906a-b92fd1c390e9 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:57:24.817537 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-unplugged", "status": "failed", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98", "code": 422}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:57:24.817836 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-021af500-5448-49a1-906a-b92fd1c390e9 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:57:24.818277 dv-cinder-791239-4-iscsi neutron-server[99041]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'code': 422} returned with failed status May 31 11:57:24.912573 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'Port': ['ID=834ff228-3f9e-460f-96e4-cd4df6eec21b,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:24.918376 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.220s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:25.061653 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Floating IP 7e57ce71-e2b7-4a92-a07c-796d6beacdf2 associated. External IP: 172.27.1.199, port: 3c46c8f8-777e-4ff2-8577-976c595eec9e. May 31 11:57:25.141079 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-004b100b-53de-4ce8-8737-cf182af2ca87 req-4a3c870e-6757-4898-a2da-967b6d8d19b2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:25.142833 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-004b100b-53de-4ce8-8737-cf182af2ca87 req-4a3c870e-6757-4898-a2da-967b6d8d19b2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.3219891 May 31 11:57:25.184631 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c63c13cb-5985-4d9d-904e-7a83ac6f13c7 req-a2763d58-679d-4c3c-80ba-b30794ce92fd tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:25.186082 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c63c13cb-5985-4d9d-904e-7a83ac6f13c7 req-a2763d58-679d-4c3c-80ba-b30794ce92fd tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0970881 May 31 11:57:25.233173 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c63c13cb-5985-4d9d-904e-7a83ac6f13c7 req-e8330310-1823-482f-b034-3394f0767458 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0411143 May 31 11:57:25.301538 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-004b100b-53de-4ce8-8737-cf182af2ca87 req-f8ed1a56-194c-4b5b-a64c-1819b40c577e tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1484373 May 31 11:57:25.428599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9cf6ea7d-f881-4b69-869e-829f6405fcba req-7fa4e079-c64e-4dcd-aee9-a98df0b8a352 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:25.430153 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9cf6ea7d-f881-4b69-869e-829f6405fcba req-7fa4e079-c64e-4dcd-aee9-a98df0b8a352 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0949955 May 31 11:57:25.482020 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9cf6ea7d-f881-4b69-869e-829f6405fcba req-18417263-8388-48a7-9f98-889832946baf tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0454531 May 31 11:57:25.625819 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Router d195a538-22a1-483f-a3e5-9007c6338ece has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:57:25.648733 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-886481a9-d04b-4637-b48c-7dd65a3f7be1 req-9154d88c-ad5d-476e-a240-f266205ae0a2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:25.650319 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-886481a9-d04b-4637-b48c-7dd65a3f7be1 req-9154d88c-ad5d-476e-a240-f266205ae0a2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.1531413 May 31 11:57:25.944761 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-886481a9-d04b-4637-b48c-7dd65a3f7be1 req-9a9c3217-5a2c-4d34-876d-9fdf481b6d0b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2264011 May 31 11:57:25.992392 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:57:26.011903 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] Sync routers for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:57:26.021660 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:57:26.022451 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:57:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:57:26.117337 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:26.164907 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:26.232814 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 3.5586417 May 31 11:57:26.234012 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:57:26.236028 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33678) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:26.241633 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-104550288', 'description': 'tempest-secgroup-smoke-104550288 description', 'project_id': '39ae37364a184ae7ae341ea7b2847c35'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:26.247947 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-90730961-a922-4f61-b161-6a877fe7923d" -d '{"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:57:26.250119 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:57:26.251074 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:26.267975 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:26.283500 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:57:26.327008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:57:26 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-43522fbc-204c-45e5-aaa3-8eca542ebd61 x-openstack-request-id: req-43522fbc-204c-45e5-aaa3-8eca542ebd61 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:57:26.327008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:57:26.327008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-43522fbc-204c-45e5-aaa3-8eca542ebd61 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:57:26.327008 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'status': 'completed', 'code': 200} May 31 11:57:26.424718 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:26.504164 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:26.511323 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Usage tracker for resource:security_group_rule and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:26.515017 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:26.526506 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:26.561276 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-90730961-a922-4f61-b161-6a877fe7923d req-f4f09b45-def0-418d-913f-a487c6bb5528 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:26.561276 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90730961-a922-4f61-b161-6a877fe7923d req-f4f09b45-def0-418d-913f-a487c6bb5528 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=39ae37364a184ae7ae341ea7b2847c35&device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1098 time: 0.2052865 May 31 11:57:26.614545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b91bfa0-d1aa-4570-841e-4ef4a7c37e9e req-2a2245c3-1e88-4541-a975-1f4b3def611f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:26.616122 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9b91bfa0-d1aa-4570-841e-4ef4a7c37e9e req-2a2245c3-1e88-4541-a975-1f4b3def611f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1614509 May 31 11:57:26.664466 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9b91bfa0-d1aa-4570-841e-4ef4a7c37e9e req-b2cf2291-5a5d-4544-a00e-8ad557209803 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0420914 May 31 11:57:26.752186 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-90730961-a922-4f61-b161-6a877fe7923d req-17f4526b-05e0-4366-97b8-d0d79f58b88c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:26.753637 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90730961-a922-4f61-b161-6a877fe7923d req-17f4526b-05e0-4366-97b8-d0d79f58b88c service neutron] 10.222.0.47 "GET /v2.0/networks?id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb HTTP/1.1" status: 200 len: 915 time: 0.1844559 May 31 11:57:26.756499 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:26.838011 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '7e57ce71-e2b7-4a92-a07c-796d6beacdf2', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'floating_ip_address': '172.27.1.199', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'port_id': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 311, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'mac_address': 'fa:16:3e:5c:72:1b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '175ced20-c6a2-4bec-82db-394e58bd641e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:57:24Z', 'updated_at': '2021-05-31T11:57:24Z', 'revision_number': 0, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:11Z', 'revision_number': 5, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:57:26.842911 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] Checking router: d195a538-22a1-483f-a3e5-9007c6338ece for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:57:26.867756 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:26.870383 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:57:26.871125 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:57:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:57:26.886538 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-684b1e44-e735-4bf4-be7f-66d58eb209a0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:26.967512 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce26793c-cc7d-4107-96c0-3a4ebb55931f req-11c66921-19a4-4b15-a0de-afbc73cc1a68 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:26.970320 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce26793c-cc7d-4107-96c0-3a4ebb55931f req-11c66921-19a4-4b15-a0de-afbc73cc1a68 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1752551 May 31 11:57:26.983461 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90730961-a922-4f61-b161-6a877fe7923d req-6c580132-96f5-46d8-b425-ecb425e3e824 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 999 time: 0.2218010 May 31 11:57:26.985165 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'SecurityGroup': ['ID=82252074-5d18-415f-aaff-3f993aa67988,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:27.090137 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce26793c-cc7d-4107-96c0-3a4ebb55931f req-007ec522-f511-400d-b764-558edd456e9d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1103518 May 31 11:57:27.148064 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.148546 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.149058 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.149428 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.149870 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.150489 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.150846 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.151278 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.151727 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.152073 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.152472 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.152864 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:57:27.174580 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.288s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:27.175335 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:27.179962 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33716) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:27.180471 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1792 time: 0.9428887 May 31 11:57:27.186382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'security_group_rule': {'security_group_id': '82252074-5d18-415f-aaff-3f993aa67988', 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:27.201734 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:27.207461 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Usage tracker for resource:security_group_rule and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:27.211359 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:27.226248 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:27.242646 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-90730961-a922-4f61-b161-6a877fe7923d req-48449732-06f9-4831-84df-eeb658045897 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:27.247408 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90730961-a922-4f61-b161-6a877fe7923d req-48449732-06f9-4831-84df-eeb658045897 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=672ec900-b55c-44a0-9ed9-7e27beeace12 HTTP/1.1" status: 200 len: 853 time: 0.2516234 May 31 11:57:27.297480 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:57:27.331998 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:27.336901 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-90730961-a922-4f61-b161-6a877fe7923d req-1db21eb6-66b5-4159-81e7-583d45a36e9a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:27.338399 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90730961-a922-4f61-b161-6a877fe7923d req-1db21eb6-66b5-4159-81e7-583d45a36e9a service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0817976 May 31 11:57:27.461412 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:27.551924 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:27.555623 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:27.573106 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90730961-a922-4f61-b161-6a877fe7923d req-dca19c26-a33e-40f6-bd41-2bb8b9ceb1b2 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2240124 May 31 11:57:27.592244 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=c7e643cf-8f5a-47e4-b9b9-187de7f7cbcc,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:27.596169 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:27.731292 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:27.733488 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-cce3755c-f7c6-4e59-aac2-8aad4e3957bc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.5517783 May 31 11:57:27.736920 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33738) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:27.747388 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'security_group_rule': {'security_group_id': '82252074-5d18-415f-aaff-3f993aa67988', 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:27.766639 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:27.776787 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Usage tracker for resource:security_group_rule and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:27.788863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:27.792866 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90730961-a922-4f61-b161-6a877fe7923d req-71cc42d3-0223-47c9-8fc0-f495ba55775d service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2115774 May 31 11:57:27.797034 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:27.842885 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '7e57ce71-e2b7-4a92-a07c-796d6beacdf2', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'floating_ip_address': '172.27.1.199', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'port_id': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 311, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'mac_address': 'fa:16:3e:5c:72:1b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '175ced20-c6a2-4bec-82db-394e58bd641e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:57:24Z', 'updated_at': '2021-05-31T11:57:24Z', 'revision_number': 0, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:11Z', 'revision_number': 5, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:57:27.849485 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a72309e9-a7ec-487b-b275-7380080e5c7d None None] The sync data for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi is: [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '7e57ce71-e2b7-4a92-a07c-796d6beacdf2', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'floating_ip_address': '172.27.1.199', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'port_id': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 311, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'mac_address': 'fa:16:3e:5c:72:1b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '175ced20-c6a2-4bec-82db-394e58bd641e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:57:24Z', 'updated_at': '2021-05-31T11:57:24Z', 'revision_number': 0, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:11Z', 'revision_number': 5, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:57:27.939422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2491c6ba-6089-4d39-a662-67a6201dc256 req-d67c2955-c53e-4df5-b739-e5868591371b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:27.940977 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2491c6ba-6089-4d39-a662-67a6201dc256 req-d67c2955-c53e-4df5-b739-e5868591371b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1248519 May 31 11:57:27.956279 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:28.104861 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a5fb684-800c-4553-b7d5-17db71b105ec req-46f4b63a-137b-47f8-a15f-65a7ceae75cb tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:28.106140 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2491c6ba-6089-4d39-a662-67a6201dc256 req-eb5a9558-4494-4a65-9816-863339a7db3c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1284769 May 31 11:57:28.118385 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9a5fb684-800c-4553-b7d5-17db71b105ec req-46f4b63a-137b-47f8-a15f-65a7ceae75cb tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1847706 May 31 11:57:28.178757 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:28.191691 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a29d385-16f7-4e52-ad25-335d6fe10907 None None] New status for floating IP 7e57ce71-e2b7-4a92-a07c-796d6beacdf2: ACTIVE {{(pid=99041) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 11:57:28.227867 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9a5fb684-800c-4553-b7d5-17db71b105ec req-bb3eb06b-44a4-497e-88ee-d87d913acafb tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1072352 May 31 11:57:28.248147 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:28.294346 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-398dcf33-579f-468e-ab00-a0fdb36f5dab tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:28.345769 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=1847a7bc-7509-4132-9406-fb8760d472e9,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:28.404426 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:28.411638 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:28.412992 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-197ab9e8-ada7-450c-b9dd-7015dd7887b2 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.6747334 May 31 11:57:28.416100 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33770) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:28.423694 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'security_group_rule': {'security_group_id': '82252074-5d18-415f-aaff-3f993aa67988', 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:28.436020 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:28.452389 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Usage tracker for resource:security_group_rule and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:28.457740 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:28.465653 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:28.517882 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e05da5d-58d6-45b0-bbbd-0f428f31620e req-a2af4237-03f0-4d4b-879e-d62f1dd6968e tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:28.528399 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4e05da5d-58d6-45b0-bbbd-0f428f31620e req-a2af4237-03f0-4d4b-879e-d62f1dd6968e tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1910939 May 31 11:57:28.578477 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00f9a35d-c75b-490d-838d-6cb951c053d5 req-b37af188-c0d7-4227-b996-23783a57e608 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:28.580000 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-00f9a35d-c75b-490d-838d-6cb951c053d5 req-b37af188-c0d7-4227-b996-23783a57e608 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1099 time: 0.2312477 May 31 11:57:28.620515 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:28.635259 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-00f9a35d-c75b-490d-838d-6cb951c053d5 req-100eba5a-f08d-48d9-8f2c-fa634c7a34c0 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0476847 May 31 11:57:28.636265 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4e05da5d-58d6-45b0-bbbd-0f428f31620e req-0f1caf74-a23a-4a18-b407-39c4e0bbb0d5 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1060135 May 31 11:57:28.722232 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:28.839125 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:28.841420 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:28.862796 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=69113212-f3af-45d3-9874-ebd64a177251,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:28.867538 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:28.931720 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33794) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:28.982116 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:28.983325 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.5661116 May 31 11:57:28.989455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33802) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:28.995128 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'security_group_rule': {'security_group_id': '82252074-5d18-415f-aaff-3f993aa67988', 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:29.022834 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:29.027145 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Usage tracker for resource:security_group_rule and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:29.030222 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:29.038558 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:29.092732 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-2a944974-e18e-471e-902e-816da6fa64ed service neutron] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.1597850 May 31 11:57:29.198869 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:29.288352 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:29.417654 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-647bc001-b619-4815-a15b-de032334e725 req-09b6b15e-9531-464d-a547-5fbf18057514 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:29.422646 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-647bc001-b619-4815-a15b-de032334e725 req-09b6b15e-9531-464d-a547-5fbf18057514 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1021111 May 31 11:57:29.452980 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:29.465182 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:29.520596 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-647bc001-b619-4815-a15b-de032334e725 req-c3c73e64-6c32-407a-b6ec-109bc1c3f621 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0893254 May 31 11:57:29.521783 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=cdbb9af5-f2f8-4a8a-b794-311b718c355a,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:29.524684 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:29.577280 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:29.581040 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c9c4142b-90b0-424d-8434-4dc31ee695e3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.5879073 May 31 11:57:29.581185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33832) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:29.586527 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'security_group_rule': {'security_group_id': '82252074-5d18-415f-aaff-3f993aa67988', 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:29.592494 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:29.596535 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Usage tracker for resource:security_group_rule and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:29.600392 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:29.612315 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:29.679022 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.securitygroups_db [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Project 39ae37364a184ae7ae341ea7b2847c35 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 31 11:57:29.704532 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:29.806459 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-346b41cd-d39e-47cb-a13f-415dddf4cbfe req-e64f9665-0f87-447a-b81b-09dc0812acb3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:29.814203 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-346b41cd-d39e-47cb-a13f-415dddf4cbfe req-e64f9665-0f87-447a-b81b-09dc0812acb3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1824803 May 31 11:57:29.818296 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:29.877217 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-346b41cd-d39e-47cb-a13f-415dddf4cbfe req-1512ce53-e01a-49bc-9c34-0843a3f63a0b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0562353 May 31 11:57:29.898723 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:29.901606 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-92302878-ff40-44ba-a0b4-ca9957cddbe0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:29.941405 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=f6d21fda-224e-4ae4-800a-87d666abc045,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:29.946816 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:30.043216 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.047017 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.4622035 May 31 11:57:30.047154 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33854) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:30.053341 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'security_group_rule': {'security_group_id': '82252074-5d18-415f-aaff-3f993aa67988', 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:30.067222 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:30.084185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Usage tracker for resource:security_group_rule and tenant:39ae37364a184ae7ae341ea7b2847c35 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:30.089590 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:30.099725 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:30.141538 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9324e27e-f04c-46d8-9aee-f64d32c280d6 req-a8ca94df-7330-450a-b85b-3c54b91c2074 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:30.143909 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9324e27e-f04c-46d8-9aee-f64d32c280d6 req-a8ca94df-7330-450a-b85b-3c54b91c2074 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1321838 May 31 11:57:30.196784 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9324e27e-f04c-46d8-9aee-f64d32c280d6 req-c287c4fa-a3f1-4d26-856f-415aded0a115 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0465114 May 31 11:57:30.260435 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.securitygroups_db [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Project 39ae37364a184ae7ae341ea7b2847c35 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 31 11:57:30.271745 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.349520 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33868) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:30.432920 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.513946 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.528439 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.531256 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:30.536791 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Usage tracker for resource:security_group_rule and tenant:ee26dffc8174474cad0f135a5856ab91 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:30.540148 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:30.546505 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:30.549810 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:30.596637 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=90a888ea-769e-4fad-87bb-7056d5683216,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:30.617670 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:30.648669 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.666403 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-48e608d7-8271-48a0-a045-4ff3d9b5cfb3 req-ec3b19a3-825f-43e8-a7ab-42a6d9d65255 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:30.666403 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-48e608d7-8271-48a0-a045-4ff3d9b5cfb3 req-ec3b19a3-825f-43e8-a7ab-42a6d9d65255 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1889544 May 31 11:57:30.690009 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.695175 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.6434236 May 31 11:57:30.722913 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-48e608d7-8271-48a0-a045-4ff3d9b5cfb3 req-2b204699-ab92-4690-987d-5e35e368b972 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0520160 May 31 11:57:30.726259 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.736387 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8e3bb02a-fa60-4553-98a0-142b8d51e464 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:30.847868 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=ee26dffc8174474cad0f135a5856ab91&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4947195 May 31 11:57:30.847868 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b18e00ee-ab42-47a1-b99d-4ec92657df23,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:30.849599 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33888) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:30.854150 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:30.886968 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-13067166-142b-4392-9174-5d8e9a906d23 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:30.892503 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-13067166-142b-4392-9174-5d8e9a906d23 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 1098 time: 0.2218769 May 31 11:57:30.907722 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22fac9a2-0e65-420b-8da0-c839dbf3803b req-cd722ceb-0d58-4e8b-b7bd-d9cf71fd6521 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:30.912300 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-22fac9a2-0e65-420b-8da0-c839dbf3803b req-cd722ceb-0d58-4e8b-b7bd-d9cf71fd6521 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1602745 May 31 11:57:30.921408 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1f2f4f91-68cb-4c9d-bc4f-5409eee6451b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:30.949929 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1f2f4f91-68cb-4c9d-bc4f-5409eee6451b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:31.042404 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-22fac9a2-0e65-420b-8da0-c839dbf3803b req-bfc70b93-6db0-43fb-a893-4630b1af3ad0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1092629 May 31 11:57:31.077723 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-87b1d6c1-b0ff-4687-aa9c-2ec8150efa76 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:31.079221 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-87b1d6c1-b0ff-4687-aa9c-2ec8150efa76 service neutron] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 912 time: 0.1690574 May 31 11:57:31.219930 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-ffce82b0-a343-4943-af21-becef059d710 req-9d7f4080-92c7-4f7d-a8f2-d02039174e0f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Synchronizing usage tracker for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:security_group {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:57:31.229090 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-6857fbf5-e397-454d-9daa-37a144851d5b service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=d447eadc-67c7-4eca-8295-e5ae670ce616 HTTP/1.1" status: 200 len: 1001 time: 0.1422274 May 31 11:57:31.296989 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1f2f4f91-68cb-4c9d-bc4f-5409eee6451b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:31.303578 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-3cb2f2fe-180b-4182-b081-be77c5403b83 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:31.305433 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-3cb2f2fe-180b-4182-b081-be77c5403b83 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0654542 May 31 11:57:31.306991 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-02b4bcb2-21b2-4086-8558-d62c34bbce2f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:31.359621 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f2f4f91-68cb-4c9d-bc4f-5409eee6451b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b18e00ee-ab42-47a1-b99d-4ec92657df23,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:31.361906 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1f2f4f91-68cb-4c9d-bc4f-5409eee6451b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:31.383562 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-351e634f-3d73-421c-b954-419e8bb89e33 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:31.384137 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1f2f4f91-68cb-4c9d-bc4f-5409eee6451b admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:31.387948 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-351e634f-3d73-421c-b954-419e8bb89e33 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0735948 May 31 11:57:31.388479 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-1f2f4f91-68cb-4c9d-bc4f-5409eee6451b admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/b18e00ee-ab42-47a1-b99d-4ec92657df23 HTTP/1.1" status: 204 len: 168 time: 0.5342462 May 31 11:57:31.413935 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-ffce82b0-a343-4943-af21-becef059d710 req-9d7f4080-92c7-4f7d-a8f2-d02039174e0f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Unset dirty status for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:security_group {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:57:31.415066 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ffce82b0-a343-4943-af21-becef059d710 req-9d7f4080-92c7-4f7d-a8f2-d02039174e0f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-104550288&fields=id&tenant_id=39ae37364a184ae7ae341ea7b2847c35 HTTP/1.1" status: 200 len: 267 time: 0.2705207 May 31 11:57:31.430630 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-230c0bdd-7998-4cde-a360-6478862cb959 req-cb16252c-1e2b-4144-9576-0c689ec4ff1e tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:31.432931 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-230c0bdd-7998-4cde-a360-6478862cb959 req-cb16252c-1e2b-4144-9576-0c689ec4ff1e tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1760545 May 31 11:57:31.439166 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd2c4296-85a2-405e-91db-f10f1535106a req-dd6de6b9-9983-4aae-910a-25bef7bd34ec service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:31.440582 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd2c4296-85a2-405e-91db-f10f1535106a req-dd6de6b9-9983-4aae-910a-25bef7bd34ec service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 1099 time: 0.1269305 May 31 11:57:31.576076 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-230c0bdd-7998-4cde-a360-6478862cb959 req-3e8d5cb3-fde4-4c9c-9c7a-d9409dc80169 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1346400 May 31 11:57:31.626183 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd2c4296-85a2-405e-91db-f10f1535106a req-fe30da33-908f-4cc1-a2a8-848391957f3c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:31.627711 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd2c4296-85a2-405e-91db-f10f1535106a req-fe30da33-908f-4cc1-a2a8-848391957f3c service neutron] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 920 time: 0.1707852 May 31 11:57:31.673917 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-b4daa44a-513a-4d34-9c4a-f473e94de297 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2814381 May 31 11:57:31.686765 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ffce82b0-a343-4943-af21-becef059d710 req-6964a523-0de4-4cf1-b0a6-3db350855753 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:31.688208 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ffce82b0-a343-4943-af21-becef059d710 req-6964a523-0de4-4cf1-b0a6-3db350855753 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.2635012 May 31 11:57:31.718040 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-ffce82b0-a343-4943-af21-becef059d710 req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'port': {'security_groups': ['7392633c-3cf7-4366-bb3e-fd78503c0a30', '82252074-5d18-415f-aaff-3f993aa67988']}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:31.806692 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd2c4296-85a2-405e-91db-f10f1535106a req-220a6eea-501e-4a04-a388-8c82a1ec13f8 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778 HTTP/1.1" status: 200 len: 217 time: 0.1676705 May 31 11:57:31.935310 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59c8961b-ff29-41c4-a2af-5ee5c08c3eac req-87b8a23e-b54e-418d-b5be-50c9bff12aad service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2461300 May 31 11:57:31.938273 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ffce82b0-a343-4943-af21-becef059d710 req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:31.939913 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd2c4296-85a2-405e-91db-f10f1535106a req-e928cec4-1abb-4103-9c2a-81be24957ca8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:31.942183 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd2c4296-85a2-405e-91db-f10f1535106a req-e928cec4-1abb-4103-9c2a-81be24957ca8 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.1222010 May 31 11:57:32.002434 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd2c4296-85a2-405e-91db-f10f1535106a req-5d6997cc-a9b7-4af4-8644-16d9b675f18b service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0492766 May 31 11:57:32.011346 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ad95cc6-a944-4c2e-a5b8-9d65dc13b78c req-5bc7a150-0727-4c67-bd3b-4488a5ddc642 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:32.012718 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2ad95cc6-a944-4c2e-a5b8-9d65dc13b78c req-5bc7a150-0727-4c67-bd3b-4488a5ddc642 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1538424 May 31 11:57:32.080104 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 33906) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:32.158871 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:32.168865 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2ad95cc6-a944-4c2e-a5b8-9d65dc13b78c req-329a0a2e-bb22-4db7-b732-e4af0c3abb76 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1323364 May 31 11:57:32.188326 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:32.195131 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd2c4296-85a2-405e-91db-f10f1535106a req-405fe490-c201-4f81-ad0f-2847a5b57c9d service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1814747 May 31 11:57:32.197068 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Usage tracker for resource:security_group_rule and tenant:2a2ad1470e4c4793bdd14957051e3ad1 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:32.201755 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:32.220751 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:32.267576 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-ffce82b0-a343-4943-af21-becef059d710 req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] For port 3c46c8f8-777e-4ff2-8577-976c595eec9e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:57:32.348119 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ffce82b0-a343-4943-af21-becef059d710 req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:32.405618 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:32.427830 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ffce82b0-a343-4943-af21-becef059d710 req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:32.436136 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:32.442644 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd2c4296-85a2-405e-91db-f10f1535106a req-b8afabf4-c3c5-41a8-9500-91b40ffc9e69 service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2397211 May 31 11:57:32.450830 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['3c46c8f8-777e-4ff2-8577-976c595eec9e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:57:32.538798 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ffce82b0-a343-4943-af21-becef059d710 req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:32.538798 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-ffce82b0-a343-4943-af21-becef059d710 req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:32.540295 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ffce82b0-a343-4943-af21-becef059d710 req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "PUT /v2.0/ports/3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 981 time: 0.8333373 May 31 11:57:32.544336 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a033658-4675-433a-b7c2-e01cace4c325 req-f4ff19fb-e123-4f78-9aad-5ced3ba4f2df tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:32.550772 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3a033658-4675-433a-b7c2-e01cace4c325 req-f4ff19fb-e123-4f78-9aad-5ced3ba4f2df tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.2247143 May 31 11:57:32.569702 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:57:32.577722 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-390e82f5-be31-4089-8c1d-5c519fb5d058" -d '{"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:57:32.588052 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:32.671194 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3a033658-4675-433a-b7c2-e01cace4c325 req-80da9467-0f18-4c43-a2cf-0e65cdd6c170 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1130099 May 31 11:57:32.698882 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=2a2ad1470e4c4793bdd14957051e3ad1&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6172738 May 31 11:57:32.701052 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 33926) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:32.709612 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['3c46c8f8-777e-4ff2-8577-976c595eec9e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:57:32.730468 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'Port': ['ID=3c46c8f8-777e-4ff2-8577-976c595eec9e,revision_number=6']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:32.759955 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.324s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:32.760596 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-014ab706-3065-482b-b83a-d3b5496a09ee admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.123s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:32.859246 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-edc27874-c207-4983-8d86-4b0cafc007a7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:32.900459 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-edc27874-c207-4983-8d86-4b0cafc007a7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:32.964555 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=da261468-e5c7-4c84-bfe7-433377410cd3,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:32.971061 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a83ce77a-9665-4dba-afcc-50d91e794032 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.210s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:32.996405 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-530527d8-430f-462f-bc4e-eef8cc552a40 None None] Provisioning complete for port 3c46c8f8-777e-4ff2-8577-976c595eec9e triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:57:32.996787 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-530527d8-430f-462f-bc4e-eef8cc552a40 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:33.003523 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d91fc90e-e404-4a02-abc9-98afd49391f7 req-a3a3b58c-1623-432b-85ab-5894babfc072 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:33.011747 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d91fc90e-e404-4a02-abc9-98afd49391f7 req-a3a3b58c-1623-432b-85ab-5894babfc072 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.2304907 May 31 11:57:33.078682 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d91fc90e-e404-4a02-abc9-98afd49391f7 req-624c589f-ea6f-4796-9c41-da0b3a353403 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0658996 May 31 11:57:33.090976 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e64ee3c8-4008-4c98-b07e-57b90c432c9f req-69d80dcd-e8d2-4acb-a0b2-a1e57b8cfec4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:33.092636 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e64ee3c8-4008-4c98-b07e-57b90c432c9f req-69d80dcd-e8d2-4acb-a0b2-a1e57b8cfec4 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1614873 May 31 11:57:33.142022 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:57:32 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-8efe0a50-daaa-4d3e-8754-6164fb19d92f x-openstack-request-id: req-8efe0a50-daaa-4d3e-8754-6164fb19d92f {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:57:33.143064 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:57:33.143715 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-8efe0a50-daaa-4d3e-8754-6164fb19d92f {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:57:33.145011 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'status': 'completed', 'code': 200} May 31 11:57:33.172676 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e64ee3c8-4008-4c98-b07e-57b90c432c9f req-57d74207-a720-4aad-9c0d-716c7c91e0af tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0604455 May 31 11:57:33.224289 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-edc27874-c207-4983-8d86-4b0cafc007a7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:33.229354 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:33.249735 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-edc27874-c207-4983-8d86-4b0cafc007a7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=da261468-e5c7-4c84-bfe7-433377410cd3,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:33.255225 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-edc27874-c207-4983-8d86-4b0cafc007a7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:33.258946 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-edc27874-c207-4983-8d86-4b0cafc007a7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:33.263038 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-edc27874-c207-4983-8d86-4b0cafc007a7 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/da261468-e5c7-4c84-bfe7-433377410cd3 HTTP/1.1" status: 204 len: 168 time: 0.5605237 May 31 11:57:33.298205 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-9072cc5e-1dae-4a43-a19c-672655ed61e5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:33.300508 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-9072cc5e-1dae-4a43-a19c-672655ed61e5 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=39ae37364a184ae7ae341ea7b2847c35&device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1137 time: 0.1212852 May 31 11:57:33.507721 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59d0766b-37a5-4dfe-a947-48be6ab82fd6 req-1d284a9b-0d13-464b-98e0-283296d12e1f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:33.510277 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59d0766b-37a5-4dfe-a947-48be6ab82fd6 req-1d284a9b-0d13-464b-98e0-283296d12e1f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1858914 May 31 11:57:33.543122 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-eea368b3-836d-4fd1-823b-7046b2a47e19 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:33.544754 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-eea368b3-836d-4fd1-823b-7046b2a47e19 service neutron] 10.222.0.47 "GET /v2.0/networks?id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb HTTP/1.1" status: 200 len: 915 time: 0.2368705 May 31 11:57:33.593344 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-59d0766b-37a5-4dfe-a947-48be6ab82fd6 req-3cadbe22-92cf-46cd-a97f-816535259d63 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0748494 May 31 11:57:33.670937 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-971be50e-f208-4761-bdcb-895f56c8d316 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 1001 time: 0.1184521 May 31 11:57:33.742084 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-b304edc7-1b8f-4ddd-84a6-580c028090f7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:33.743625 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-b304edc7-1b8f-4ddd-84a6-580c028090f7 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=672ec900-b55c-44a0-9ed9-7e27beeace12 HTTP/1.1" status: 200 len: 853 time: 0.0661023 May 31 11:57:33.829190 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-d25e9811-514f-4497-b3ce-14192e3d249c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:33.830824 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-d25e9811-514f-4497-b3ce-14192e3d249c service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0788872 May 31 11:57:34.040127 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b624d954-e8e6-4bf6-9983-4f15420a44b0 req-8e1eeb19-7ae5-4682-b933-66da587742a6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:34.041654 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b624d954-e8e6-4bf6-9983-4f15420a44b0 req-8e1eeb19-7ae5-4682-b933-66da587742a6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.2061725 May 31 11:57:34.045420 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-3dc62ca0-6aac-43ac-8512-c34723b5e609 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1783175 May 31 11:57:34.253353 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b624d954-e8e6-4bf6-9983-4f15420a44b0 req-bec677ee-d85f-4ade-b195-ee0789e9f7c9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2057362 May 31 11:57:34.296933 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-390e82f5-be31-4089-8c1d-5c519fb5d058 req-c0f55aeb-772f-4b23-b94a-b472f4e607c6 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2339084 May 31 11:57:34.375948 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-44f9eb70-a849-4d63-8252-d0c33c3e98c0 req-5a9e8d46-7690-47f4-b4d8-0c27b729b8c3 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:34.377699 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-44f9eb70-a849-4d63-8252-d0c33c3e98c0 req-5a9e8d46-7690-47f4-b4d8-0c27b729b8c3 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0898514 May 31 11:57:34.430228 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-44f9eb70-a849-4d63-8252-d0c33c3e98c0 req-5aed5337-fa84-48d5-8f63-5fa1a4858f21 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0453312 May 31 11:57:35.622835 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a5b074-dd68-4328-9402-c5819fc35080 req-be68602c-fc9b-4ab2-a7fe-8124b5f7afe9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:35.624335 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-29a5b074-dd68-4328-9402-c5819fc35080 req-be68602c-fc9b-4ab2-a7fe-8124b5f7afe9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1008003 May 31 11:57:35.723633 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-29a5b074-dd68-4328-9402-c5819fc35080 req-b8c7b28f-700a-4040-b6d8-cb1dbf3447fb tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0857720 May 31 11:57:35.726593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e990e1f7-20f2-4ca4-88f9-42f60cf05e41 req-94b7f20e-fb4a-4480-8e57-375adb7b2320 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:35.736061 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e990e1f7-20f2-4ca4-88f9-42f60cf05e41 req-94b7f20e-fb4a-4480-8e57-375adb7b2320 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1362813 May 31 11:57:35.786401 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e990e1f7-20f2-4ca4-88f9-42f60cf05e41 req-809f8b5e-1c18-4162-ba9d-326a01563915 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0485291 May 31 11:57:36.172462 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34012) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:36.301522 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:36.317512 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:36.321490 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Usage tracker for resource:security_group_rule and tenant:587fcc61b815440a96f295f5af106c6d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:36.324418 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:36.330225 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:36.412616 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:36.650812 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:36.658180 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-3d8b2ab8-6f7c-4d0e-9447-efc03a3eabfd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:36.698300 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1861c46c-779e-4966-9866-afc84e080b9d,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:36.702631 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=587fcc61b815440a96f295f5af106c6d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5291431 May 31 11:57:36.703567 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:36.705266 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34032) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:36.774824 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-991e5232-f117-482b-ac23-429226170658 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:36.804314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-991e5232-f117-482b-ac23-429226170658 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:36.959074 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6f758d8-fa56-4881-b0bc-eb79431ff125 req-bc57231b-1b4a-4254-803b-943c311e14b8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:36.959074 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6f758d8-fa56-4881-b0bc-eb79431ff125 req-bc57231b-1b4a-4254-803b-943c311e14b8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0891602 May 31 11:57:37.007483 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6f758d8-fa56-4881-b0bc-eb79431ff125 req-71918deb-62e9-47cc-9698-8adb652b6a8a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0407047 May 31 11:57:37.021656 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-991e5232-f117-482b-ac23-429226170658 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.026988 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-49263a23-98a9-40c0-922c-c4da1ab9e9ea admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:37.041065 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-991e5232-f117-482b-ac23-429226170658 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1861c46c-779e-4966-9866-afc84e080b9d,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:37.044680 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-991e5232-f117-482b-ac23-429226170658 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:37.052737 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-991e5232-f117-482b-ac23-429226170658 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.056405 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-991e5232-f117-482b-ac23-429226170658 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/1861c46c-779e-4966-9866-afc84e080b9d HTTP/1.1" status: 204 len: 168 time: 0.3475630 May 31 11:57:37.175080 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34046) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:37.291024 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.309067 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:37.313981 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Usage tracker for resource:security_group_rule and tenant:850f42d7683a4290884e88b613418609 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:37.317293 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:37.323374 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:37.408910 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.488769 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.511714 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1f2f4f91-68cb-4c9d-bc4f-5409eee6451b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:37.554438 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=850f42d7683a4290884e88b613418609&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3781829 May 31 11:57:37.556005 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9f576cf3-c3ff-4668-8442-b5982366b3de,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:37.557077 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34058) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:37.564560 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:37.622648 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-75b37657-8865-4ed1-8297-5919c528be35 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.649829 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-75b37657-8865-4ed1-8297-5919c528be35 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.779667 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-75b37657-8865-4ed1-8297-5919c528be35 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.786547 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fbe615c9-b848-4cbd-9b8a-a727d13ab8a6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:37.799646 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-75b37657-8865-4ed1-8297-5919c528be35 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9f576cf3-c3ff-4668-8442-b5982366b3de,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:37.803050 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-75b37657-8865-4ed1-8297-5919c528be35 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:37.804480 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-75b37657-8865-4ed1-8297-5919c528be35 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:37.808830 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-75b37657-8865-4ed1-8297-5919c528be35 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/9f576cf3-c3ff-4668-8442-b5982366b3de HTTP/1.1" status: 204 len: 168 time: 0.2476456 May 31 11:57:38.223945 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2d8665f7-865f-4881-a214-c62bebb60dfd req-2cacf0aa-e39f-43d7-9c03-aa5650e1b80b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:38.225447 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2d8665f7-865f-4881-a214-c62bebb60dfd req-2cacf0aa-e39f-43d7-9c03-aa5650e1b80b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0870869 May 31 11:57:38.307932 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2d8665f7-865f-4881-a214-c62bebb60dfd req-ec3d0a37-9704-410a-87da-b63eadaa0413 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0741179 May 31 11:57:38.450900 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-eb0b090f-dcc3-4a17-8159-560be80466d7 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0727632 May 31 11:57:38.572432 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-bac5870f-3891-4aee-b4dd-21b48d8f2b43 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:38.573955 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-bac5870f-3891-4aee-b4dd-21b48d8f2b43 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=c94978b38250481095d2824adfb16ed4&device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 1098 time: 0.0752001 May 31 11:57:38.643980 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34096) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:38.663812 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:38.679456 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:38.683610 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Usage tracker for resource:security_group_rule and tenant:c00a6d6c678f4fbab36c5bdf3a0759c6 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:38.686746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:38.693896 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:38.748786 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-f4bd5f6b-db8e-47f8-b233-77b070daf6c0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:38.750334 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-f4bd5f6b-db8e-47f8-b233-77b070daf6c0 service neutron] 10.222.0.47 "GET /v2.0/networks?id=c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 200 len: 920 time: 0.1402767 May 31 11:57:38.782984 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:38.829527 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-abd0af73-b656-4cf3-8bab-1159b91c7818 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e HTTP/1.1" status: 200 len: 217 time: 0.0687323 May 31 11:57:38.901343 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-6deff700-63c5-42cf-bdef-a6ebe072378f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:38.902309 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:38.902813 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-6deff700-63c5-42cf-bdef-a6ebe072378f service neutron] 10.222.0.47 "GET /v2.0/subnets?id=6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 200 len: 859 time: 0.0661681 May 31 11:57:38.917192 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-2a661d68-43b8-451f-91a1-437b9b59e468 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.002s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:38.963301 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=c00a6d6c678f4fbab36c5bdf3a0759c6&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3175550 May 31 11:57:38.964494 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bac057bc-b156-41ca-8a08-a384a39201d1,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:38.966830 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34104) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:38.969307 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:38.976171 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-0c25e0bf-211b-4f21-b82f-b53fde0e79b5 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=c36bb4a9-2822-49b1-b291-69604a881999&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0608404 May 31 11:57:38.979482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:39.027152 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c2441448-6aba-4fb0-bde5-05172ab7d427 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:39.054100 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c2441448-6aba-4fb0-bde5-05172ab7d427 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:39.113842 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-b2659497-c9f4-45e7-b483-483735b5301e service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1292155 May 31 11:57:39.254919 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6ebb3ad6-7d7d-4966-aeff-ee273312ad21 req-6395d1ca-4ac4-430e-a001-9c83e577f15b service neutron] 10.222.0.47 "GET /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1326993 May 31 11:57:39.305290 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c2441448-6aba-4fb0-bde5-05172ab7d427 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:39.310313 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6a52b119-e4f2-4a80-812b-34c2ea3c9a2e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:39.324754 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2441448-6aba-4fb0-bde5-05172ab7d427 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bac057bc-b156-41ca-8a08-a384a39201d1,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:39.327289 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c2441448-6aba-4fb0-bde5-05172ab7d427 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:39.328475 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c2441448-6aba-4fb0-bde5-05172ab7d427 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:39.329673 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c2441448-6aba-4fb0-bde5-05172ab7d427 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/bac057bc-b156-41ca-8a08-a384a39201d1 HTTP/1.1" status: 204 len: 168 time: 0.3618326 May 31 11:57:39.549278 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca0458fd-4ea7-426a-a797-e7ae0fd69e54 req-2d5e9bec-c402-4bdb-96ae-31e1807984ac tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:39.549278 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca0458fd-4ea7-426a-a797-e7ae0fd69e54 req-2d5e9bec-c402-4bdb-96ae-31e1807984ac tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0903916 May 31 11:57:39.587896 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca0458fd-4ea7-426a-a797-e7ae0fd69e54 req-f94fe8ad-d347-403f-88a7-bf71527e1f04 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393059 May 31 11:57:40.830163 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ce31542-cca8-4618-9617-f2898bef4076 req-40e11c79-a8a8-4087-8c7a-8685788f7265 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:40.833543 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-40e11c79-a8a8-4087-8c7a-8685788f7265 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17&tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d HTTP/1.1" status: 200 len: 1096 time: 0.1884694 May 31 11:57:40.840139 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a7a5cc8-737a-43c7-a73a-8e46d9d273af req-15b89f17-c828-4240-80aa-343c61da04ae tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:40.847394 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6a7a5cc8-737a-43c7-a73a-8e46d9d273af req-15b89f17-c828-4240-80aa-343c61da04ae tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1309686 May 31 11:57:40.936781 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6a7a5cc8-737a-43c7-a73a-8e46d9d273af req-73a2ac47-6042-4990-bd95-d1610d7f0a4b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0883455 May 31 11:57:40.967088 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ce31542-cca8-4618-9617-f2898bef4076 req-776c2aac-8088-42b4-84e1-d6209436b9e2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:40.968648 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-776c2aac-8088-42b4-84e1-d6209436b9e2 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 1096 time: 0.1017241 May 31 11:57:41.096769 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-6916cd0a-afc2-48cb-9fa1-2d23acf949f2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/networks?tenant_id=d4b0a729336241e29d7d74dcbc6dc060&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0357969 May 31 11:57:41.122661 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ce31542-cca8-4618-9617-f2898bef4076 req-20a397f3-9fb4-4d2b-9226-d22b5f0a1d47 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:41.124144 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-20a397f3-9fb4-4d2b-9226-d22b5f0a1d47 service neutron] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 917 time: 0.1474581 May 31 11:57:41.336525 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-1fd6f2ba-3261-41b5-8122-7763e2dd5ce7 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98 HTTP/1.1" status: 200 len: 999 time: 0.2011561 May 31 11:57:41.359261 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-a8a9aca4-1767-4c2a-a85b-b66298881949 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:41.360827 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-a8a9aca4-1767-4c2a-a85b-b66298881949 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.2574141 May 31 11:57:41.377243 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-ef8b7178-8c36-423c-8afa-60ccf39f6027 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/quotas/d4b0a729336241e29d7d74dcbc6dc060 HTTP/1.1" status: 200 len: 375 time: 0.0095561 May 31 11:57:41.409007 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ce31542-cca8-4618-9617-f2898bef4076 req-9e30b74d-e974-4530-a4b1-aab8ca9fcf4e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:41.410426 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-9e30b74d-e974-4530-a4b1-aab8ca9fcf4e service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0606089 May 31 11:57:41.431592 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-0de14bfa-10d7-41e7-86c9-9d6c7ef5f40d tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=d4b0a729336241e29d7d74dcbc6dc060&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0477421 May 31 11:57:41.503407 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ce31542-cca8-4618-9617-f2898bef4076 req-ce5ce7b9-b5d3-43f9-a0b7-4665299a2994 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:41.503407 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-ce5ce7b9-b5d3-43f9-a0b7-4665299a2994 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0773971 May 31 11:57:41.640999 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-3cbf25d7-e716-47e0-ac0a-16673fe19f0e service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1306491 May 31 11:57:41.702762 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34176) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:41.787672 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a516af66-11be-414e-acdd-7093217d8ccc req-1bfa7c07-f3b7-4101-bdb9-5124a882415d tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0617728 May 31 11:57:41.856370 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-0a1addab-adf3-4380-8127-2f254600a201 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2083519 May 31 11:57:41.861627 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:41.878380 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:41.882703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Usage tracker for resource:security_group_rule and tenant:acc9b557be58456aa8dcbdb39f4e1688 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:41.885685 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:41.891623 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:41.973112 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:42.131873 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:42.147995 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-53d720e8-9792-4e9b-ae28-4295b5f0b23b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:42.200779 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=acc9b557be58456aa8dcbdb39f4e1688&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4967430 May 31 11:57:42.203277 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34202) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:42.203652 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bfcc9ce2-497e-4ab7-8d77-ccd7d571c556,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:42.208061 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:42.232865 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-09105923-2920-4275-9e85-ca7b2a6e899a req-dd32a006-5267-48d7-9e77-745b3e55b841 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:42.234690 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-09105923-2920-4275-9e85-ca7b2a6e899a req-dd32a006-5267-48d7-9e77-745b3e55b841 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1436911 May 31 11:57:42.272873 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-225c1b8a-22d1-4a23-ae12-fe0b65a23e00 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:42.284098 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-09105923-2920-4275-9e85-ca7b2a6e899a req-f327d44f-d90f-4c34-9234-ddf23b0953c6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0427408 May 31 11:57:42.309741 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-225c1b8a-22d1-4a23-ae12-fe0b65a23e00 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:42.425306 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-225c1b8a-22d1-4a23-ae12-fe0b65a23e00 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:42.434705 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-edc27874-c207-4983-8d86-4b0cafc007a7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:42.453648 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-225c1b8a-22d1-4a23-ae12-fe0b65a23e00 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bfcc9ce2-497e-4ab7-8d77-ccd7d571c556,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:42.457544 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-225c1b8a-22d1-4a23-ae12-fe0b65a23e00 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:42.468151 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-225c1b8a-22d1-4a23-ae12-fe0b65a23e00 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:42.469887 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-225c1b8a-22d1-4a23-ae12-fe0b65a23e00 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/bfcc9ce2-497e-4ab7-8d77-ccd7d571c556 HTTP/1.1" status: 204 len: 168 time: 0.2654536 May 31 11:57:42.945364 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4bfad883-45cd-4ef1-86f1-b0a2a466d28b req-a780293c-e0e4-4dad-9be5-63990edc69c0 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0674326 May 31 11:57:43.564540 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-84bb322c-9818-4f9e-b6b3-34d46ad51c7f req-01a449ee-dff5-48b4-a768-aab9b9404f8e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:43.566263 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-84bb322c-9818-4f9e-b6b3-34d46ad51c7f req-01a449ee-dff5-48b4-a768-aab9b9404f8e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1406715 May 31 11:57:43.636147 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-84bb322c-9818-4f9e-b6b3-34d46ad51c7f req-ca9bc56f-29a9-4045-99cb-3a9fbf33c802 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0640032 May 31 11:57:44.346839 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6d203a92-fea1-4cb3-8e6e-d86bcfc8c37c req-b9a0c10f-6c60-4fa4-9304-0241dc04ddc2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0554676 May 31 11:57:44.415729 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdb512b3-b3e1-4dd3-bd62-7637e3b349ce req-61bf4e30-c2c2-4dfd-98a3-fa5e7cdfce98 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:44.417221 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdb512b3-b3e1-4dd3-bd62-7637e3b349ce req-61bf4e30-c2c2-4dfd-98a3-fa5e7cdfce98 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0925403 May 31 11:57:44.467714 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fdb512b3-b3e1-4dd3-bd62-7637e3b349ce req-37d0c9b7-615e-4033-aba9-c2ef35444c13 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0444398 May 31 11:57:44.484336 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:57:44.565603 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1238e569-7bb5-40bc-a38e-da97acbd30cc req-80748042-8cd6-4aa7-a3a6-f48079fb8a77 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:44.568188 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1238e569-7bb5-40bc-a38e-da97acbd30cc req-80748042-8cd6-4aa7-a3a6-f48079fb8a77 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1209207 May 31 11:57:44.615992 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1238e569-7bb5-40bc-a38e-da97acbd30cc req-32745177-f922-49fd-a757-2112f886e485 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416217 May 31 11:57:44.743503 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34278) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:44.873516 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:44.892946 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:44.899589 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Usage tracker for resource:security_group_rule and tenant:792ddf24301f49e4a4ede657dfb08022 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:44.910739 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c97e5ba-771f-447e-8c15-6f4648141423 req-d679bdcf-7c12-4d1d-a34f-5a262e808a55 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:44.912253 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1c97e5ba-771f-447e-8c15-6f4648141423 req-d679bdcf-7c12-4d1d-a34f-5a262e808a55 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1412678 May 31 11:57:44.913154 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:44.920453 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:44.979711 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1c97e5ba-771f-447e-8c15-6f4648141423 req-02d3b0b6-2776-4bd7-ad5e-23806d99baca tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0603020 May 31 11:57:45.043477 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:45.130567 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:45.137705 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-666c4119-8e3a-4dda-9807-41047429d610 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:45.179169 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=792ddf24301f49e4a4ede657dfb08022&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4345808 May 31 11:57:45.180131 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=16dd6a2b-0ba1-4cce-bd15-af9fd5916e1a,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:45.181867 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34284) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:57:45.183253 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:45.246037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-57f19dca-da12-4ce6-bea6-c876fe67a29f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:45.272189 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-57f19dca-da12-4ce6-bea6-c876fe67a29f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:45.439060 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-57f19dca-da12-4ce6-bea6-c876fe67a29f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:45.443946 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f649a3b3-dae7-43cc-98fb-225c2d003f15 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:45.454140 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-57f19dca-da12-4ce6-bea6-c876fe67a29f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=16dd6a2b-0ba1-4cce-bd15-af9fd5916e1a,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:45.457602 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-57f19dca-da12-4ce6-bea6-c876fe67a29f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:45.496638 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-57f19dca-da12-4ce6-bea6-c876fe67a29f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:45.497526 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-57f19dca-da12-4ce6-bea6-c876fe67a29f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/16dd6a2b-0ba1-4cce-bd15-af9fd5916e1a HTTP/1.1" status: 204 len: 168 time: 0.3145857 May 31 11:57:45.512994 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4db68215-a245-4c99-809c-a18fa298af92 req-78c297b1-22c0-4866-844b-fb5b1e6e6377 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0533242 May 31 11:57:45.853620 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e396c221-d9bc-49f4-8b84-4e37db959b8e req-40d06de3-3c5a-4113-9438-ceab23c98c2f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:45.858507 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50e33128-1f49-4100-9285-206254d5acaa req-41a4234b-f702-4463-8b3f-331d5569b292 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:45.859681 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e396c221-d9bc-49f4-8b84-4e37db959b8e req-40d06de3-3c5a-4113-9438-ceab23c98c2f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0920908 May 31 11:57:45.860359 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-50e33128-1f49-4100-9285-206254d5acaa req-41a4234b-f702-4463-8b3f-331d5569b292 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1024680 May 31 11:57:45.930452 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e396c221-d9bc-49f4-8b84-4e37db959b8e req-e66c7e79-e391-43ce-9771-b21ae85ff076 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0566392 May 31 11:57:45.940626 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-50e33128-1f49-4100-9285-206254d5acaa req-c8570e52-6dfe-4b12-935a-1dd5df594d6a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0623407 May 31 11:57:46.187245 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d11c014c-65c7-4a26-9bb5-fb69704ab5d8 req-be2c7c80-bc5e-4953-9c26-67a7f3e1b5d9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:46.188762 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d11c014c-65c7-4a26-9bb5-fb69704ab5d8 req-be2c7c80-bc5e-4953-9c26-67a7f3e1b5d9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0847237 May 31 11:57:46.243822 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d11c014c-65c7-4a26-9bb5-fb69704ab5d8 req-6eec62e3-054a-4266-bd38-8338a4b54128 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0485251 May 31 11:57:46.669127 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-27f753ba-98e9-4780-b01c-fd07b7fab51e req-74cf72b2-8496-4721-8db0-9a04e0e1e628 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0456371 May 31 11:57:47.170437 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45181589-4688-41c8-a46a-041f2a0a9f89 req-6af8a98c-d435-493f-95a9-f41087e701c2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:47.171865 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-45181589-4688-41c8-a46a-041f2a0a9f89 req-6af8a98c-d435-493f-95a9-f41087e701c2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.1232107 May 31 11:57:47.195991 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4401195-ec8a-4b74-8664-c3225c80c2d7 req-102f5a47-a182-49e9-bc1b-3c31a632cd49 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:47.203287 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f4401195-ec8a-4b74-8664-c3225c80c2d7 req-102f5a47-a182-49e9-bc1b-3c31a632cd49 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1229801 May 31 11:57:47.238530 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-45181589-4688-41c8-a46a-041f2a0a9f89 req-c35601c3-88af-4621-b18c-00065833a62c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0602334 May 31 11:57:47.266236 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f4401195-ec8a-4b74-8664-c3225c80c2d7 req-008ac4bb-6272-44bd-9e0d-e35d106d5bda tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0619233 May 31 11:57:47.468223 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-519dedd8-ad59-4a84-84ab-c560ee9ae825 req-9d86f38d-4a4c-4042-8ed0-bef9906deb6c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:47.469958 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-519dedd8-ad59-4a84-84ab-c560ee9ae825 req-9d86f38d-4a4c-4042-8ed0-bef9906deb6c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0913727 May 31 11:57:47.521226 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-519dedd8-ad59-4a84-84ab-c560ee9ae825 req-2660e022-7864-4706-81d9-f17a04546642 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0451775 May 31 11:57:47.828675 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b041706d-3df6-4ce2-89ce-9509a0725892 req-599b776f-f10b-4f18-8f92-6ba220c07b9a tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0463078 May 31 11:57:48.456625 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68ecf36c-f616-462b-bc45-6e7c9ca673b4 req-6475ff32-01b6-4a00-a230-0ee7c3dc162c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:48.459102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76944485-652b-4d66-94a2-42023de1b1de req-754a01c5-d610-45d7-bc58-236863ff7fff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:48.459236 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-68ecf36c-f616-462b-bc45-6e7c9ca673b4 req-6475ff32-01b6-4a00-a230-0ee7c3dc162c tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.1094661 May 31 11:57:48.460525 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-76944485-652b-4d66-94a2-42023de1b1de req-754a01c5-d610-45d7-bc58-236863ff7fff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0830898 May 31 11:57:48.507890 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-76944485-652b-4d66-94a2-42023de1b1de req-9f7d29fa-4a45-472e-8721-44a8c101677a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0418591 May 31 11:57:48.510550 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-68ecf36c-f616-462b-bc45-6e7c9ca673b4 req-66538701-8334-42ef-8cc4-f71e1803fc8b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0450039 May 31 11:57:48.719072 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8faa58aa-49c2-4afc-9e14-25f1fb2b67e1 req-49758996-dcde-4ac1-a0c2-2512b563db59 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:48.720488 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8faa58aa-49c2-4afc-9e14-25f1fb2b67e1 req-49758996-dcde-4ac1-a0c2-2512b563db59 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0829251 May 31 11:57:48.763278 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8faa58aa-49c2-4afc-9e14-25f1fb2b67e1 req-f2edefba-b9aa-467c-b826-decbd4533344 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0364633 May 31 11:57:48.991156 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e2d13095-7270-4fc2-91cf-fac9ab714a7b req-0358fa8b-849b-47c5-bbbe-0cf557145aa8 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0529923 May 31 11:57:49.722713 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-04fd6021-a599-49e7-86f1-62807fca98b3 req-726f3d41-c4f6-4957-aad5-8446246f8a48 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:49.724178 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-04fd6021-a599-49e7-86f1-62807fca98b3 req-726f3d41-c4f6-4957-aad5-8446246f8a48 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1038465 May 31 11:57:49.745935 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10d118d4-8384-4722-a313-5addf5fbb83b req-9c5a6384-f635-45a0-816a-4c982a406a87 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:49.747770 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-10d118d4-8384-4722-a313-5addf5fbb83b req-9c5a6384-f635-45a0-816a-4c982a406a87 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0898247 May 31 11:57:49.782898 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-04fd6021-a599-49e7-86f1-62807fca98b3 req-3868b278-4490-4e1e-b3db-3954cdf366ac tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0525551 May 31 11:57:49.809782 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-10d118d4-8384-4722-a313-5addf5fbb83b req-917bccad-fd3b-46da-a5a0-f6884deac23b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0536127 May 31 11:57:49.965625 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dc50c3b-4a31-401d-9350-46a2dee5bd39 req-f13bb512-534a-4af7-9adc-8b59cbe466e1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:49.967064 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4dc50c3b-4a31-401d-9350-46a2dee5bd39 req-f13bb512-534a-4af7-9adc-8b59cbe466e1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0884392 May 31 11:57:50.011913 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4dc50c3b-4a31-401d-9350-46a2dee5bd39 req-91136157-a62d-4cdb-9cd1-98fe0a3adced tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0383916 May 31 11:57:50.149662 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2ae77454-d0ae-44b1-94fa-cc97bb163a4c req-a734b6ac-cf16-489c-b455-898202dbbcec tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0456200 May 31 11:57:50.884371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ab17d374-000f-490b-a8a0-ee998e88f3ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:51.011515 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6929b875-6f17-466b-a854-703e29eb895e req-281e441a-0a2b-4464-9605-c410f1aa3725 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:51.014078 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6929b875-6f17-466b-a854-703e29eb895e req-281e441a-0a2b-4464-9605-c410f1aa3725 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0911965 May 31 11:57:51.072915 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6929b875-6f17-466b-a854-703e29eb895e req-88d9ae55-4df2-4168-9207-2269766bffb4 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0521028 May 31 11:57:51.093888 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4005cb8-d28f-4222-8c59-9a67ba1fc448 req-d129d4e1-2c92-45a3-8f54-683c6f313ea7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:51.095415 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a4005cb8-d28f-4222-8c59-9a67ba1fc448 req-d129d4e1-2c92-45a3-8f54-683c6f313ea7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.1773806 May 31 11:57:51.143877 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a4005cb8-d28f-4222-8c59-9a67ba1fc448 req-f166347c-74e9-4f7c-9025-a3f90782ba44 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0421503 May 31 11:57:51.264688 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5cd32bee-0f01-429c-bcee-b8a053a16bb1 req-d840dacd-220c-4ca9-9d70-da6ca26e7b49 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:51.266184 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5cd32bee-0f01-429c-bcee-b8a053a16bb1 req-d840dacd-220c-4ca9-9d70-da6ca26e7b49 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0871017 May 31 11:57:51.320410 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-164d85ea-60a6-4e0f-89ed-38918de8d12a req-331729b5-c281-4cf0-9ccf-03cd06e45745 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0584242 May 31 11:57:51.320668 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5cd32bee-0f01-429c-bcee-b8a053a16bb1 req-e4e093fc-c0e2-4167-bd36-103edea6f6c5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475104 May 31 11:57:52.295070 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b330d970-e278-4aff-b463-fe5567433fd7 req-e5a3738c-70da-41f7-822b-258cc7c6b16a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:52.297274 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b330d970-e278-4aff-b463-fe5567433fd7 req-e5a3738c-70da-41f7-822b-258cc7c6b16a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.1034327 May 31 11:57:52.347914 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b330d970-e278-4aff-b463-fe5567433fd7 req-ca6169b4-c986-4dc9-8e39-887308bc41b5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0441787 May 31 11:57:52.351847 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4890498-176d-46ef-ad54-452b6e11a9f3 req-83788425-81b3-4345-8498-5648e5c3969e tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:52.353365 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e4890498-176d-46ef-ad54-452b6e11a9f3 req-83788425-81b3-4345-8498-5648e5c3969e tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0879397 May 31 11:57:52.407088 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e4890498-176d-46ef-ad54-452b6e11a9f3 req-74d2e5ee-9561-4249-bc7d-37932c8b0bfa tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0477901 May 31 11:57:52.492760 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b3d9b03d-a6fd-493c-a85e-dfdfa31f29cd req-6a57481d-b8d7-46db-93c1-3f3966bf72b7 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0458207 May 31 11:57:52.662996 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f6722a6-e0ac-43cc-81a4-76203e3c3272 req-7373a7ed-24e3-4449-ad3e-4976a33b342b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:52.664284 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6f6722a6-e0ac-43cc-81a4-76203e3c3272 req-7373a7ed-24e3-4449-ad3e-4976a33b342b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1560802 May 31 11:57:52.711208 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6f6722a6-e0ac-43cc-81a4-76203e3c3272 req-fa93a938-cfac-45c8-b8db-d4c8851eb18b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0407014 May 31 11:57:53.549876 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d9b9b48-1596-4830-85a3-8d6d514e9e1d req-f2a2d239-f266-4351-b1f9-89147dd0a266 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:53.552093 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1d9b9b48-1596-4830-85a3-8d6d514e9e1d req-f2a2d239-f266-4351-b1f9-89147dd0a266 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0793886 May 31 11:57:53.570565 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-30fe80ab-7d83-4f20-92c5-99864d60346c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:53.572027 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:53.600752 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1d9b9b48-1596-4830-85a3-8d6d514e9e1d req-ee78a98a-da8d-45da-b7cc-d3790276aab2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0429821 May 31 11:57:53.795439 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2949d009-beb0-4d8b-97bc-05afae878e0e req-731dade9-9f91-42b4-b796-38901aadc7cf tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0426707 May 31 11:57:53.797152 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0b6f9fa-dc4a-425e-8837-818aa6d04075 req-f06de8b3-e103-40ac-834f-228d8efb5299 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:53.798595 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d0b6f9fa-dc4a-425e-8837-818aa6d04075 req-f06de8b3-e103-40ac-834f-228d8efb5299 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0865133 May 31 11:57:53.865300 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d0b6f9fa-dc4a-425e-8837-818aa6d04075 req-a1729fd6-2127-4a5e-a892-ef353ba83bf8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0603700 May 31 11:57:53.994262 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b89e60a-f26a-4a09-ad37-1a00fb7a9200 req-8e9d97bd-2cf2-4fde-ab95-5f89a09e0ec7 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:53.995950 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5b89e60a-f26a-4a09-ad37-1a00fb7a9200 req-8e9d97bd-2cf2-4fde-ab95-5f89a09e0ec7 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0951796 May 31 11:57:54.061838 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-344acfe1-6a2c-478f-89b5-51e74f77743b req-0829cdf5-eaeb-4860-b1c9-ecd6ae575096 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:54.063343 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-344acfe1-6a2c-478f-89b5-51e74f77743b req-0829cdf5-eaeb-4860-b1c9-ecd6ae575096 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0989661 May 31 11:57:54.066701 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5b89e60a-f26a-4a09-ad37-1a00fb7a9200 req-a15832d4-1923-4bba-ba1c-5e5bd49cc86b tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0645473 May 31 11:57:54.108815 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-344acfe1-6a2c-478f-89b5-51e74f77743b req-fc900472-2dd2-4cc1-9378-64df7078bb9f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0391777 May 31 11:57:54.593175 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:54.808118 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0d067ba-42f0-4b5f-b273-efef4936a29e req-d0cb9e12-b523-4640-8431-322d6c7f9c09 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:54.809534 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0d067ba-42f0-4b5f-b273-efef4936a29e req-d0cb9e12-b523-4640-8431-322d6c7f9c09 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0826786 May 31 11:57:54.858818 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d0d067ba-42f0-4b5f-b273-efef4936a29e req-3cd04fdf-2faf-4c10-ae1a-c3c8b9cae332 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0435448 May 31 11:57:54.953837 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-26f00968-0936-4267-9bd9-99830a9ec8e1 req-48cca8e3-1ded-4e64-b290-4e7f2da672de tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0466735 May 31 11:57:55.068916 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-272a9064-6461-46b1-9ceb-e092436d92b2 req-8e200b1c-02d9-473e-8f49-6edd74b39d7d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:55.070365 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-272a9064-6461-46b1-9ceb-e092436d92b2 req-8e200b1c-02d9-473e-8f49-6edd74b39d7d tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0915556 May 31 11:57:55.119356 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-272a9064-6461-46b1-9ceb-e092436d92b2 req-c74bc462-6380-49c0-9ee4-5bd842098619 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422971 May 31 11:57:55.266466 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-60771fe5-4481-47d0-b9a2-876565fca9eb req-bb5ee441-c226-4f90-a5d7-51665b3996fe tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:55.271566 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-60771fe5-4481-47d0-b9a2-876565fca9eb req-bb5ee441-c226-4f90-a5d7-51665b3996fe tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0919487 May 31 11:57:55.320068 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-60771fe5-4481-47d0-b9a2-876565fca9eb req-f5b29bb2-19bf-473b-9a85-3e852ec0d422 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0439522 May 31 11:57:55.339169 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ddf9e46-46af-4221-89c4-84efa02ec589 req-198b4c90-1492-4e97-963b-95633c83b39e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:55.340598 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5ddf9e46-46af-4221-89c4-84efa02ec589 req-198b4c90-1492-4e97-963b-95633c83b39e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0983908 May 31 11:57:55.390714 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5ddf9e46-46af-4221-89c4-84efa02ec589 req-5be91e62-b460-4422-8ec7-b8393200a7d0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0443790 May 31 11:57:56.068497 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66c8f7ce-3894-4dde-b624-71e41dd8e724 req-592c417f-7b6f-4494-b25c-c41f557661fa tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:56.070084 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c8f7ce-3894-4dde-b624-71e41dd8e724 req-592c417f-7b6f-4494-b25c-c41f557661fa tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0991051 May 31 11:57:56.101330 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:56.115252 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ceb2d90-9336-4a59-9406-3680ec5a607f req-cf7d999a-8a53-45cc-8685-f0f0bcc4b488 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0454042 May 31 11:57:56.118811 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-66c8f7ce-3894-4dde-b624-71e41dd8e724 req-640ba307-6acd-4f10-953b-943945e7dfeb tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416648 May 31 11:57:56.329904 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93a20ebe-460d-4c75-ad5c-cf381a841b9c req-8d11a545-15d4-4325-9b97-5cb909bf15bf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:56.331280 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-93a20ebe-460d-4c75-ad5c-cf381a841b9c req-8d11a545-15d4-4325-9b97-5cb909bf15bf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0964499 May 31 11:57:56.386751 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-93a20ebe-460d-4c75-ad5c-cf381a841b9c req-67e5f706-50d5-41f7-8f73-c3cc6d9da537 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0485508 May 31 11:57:56.549543 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b644436c-bbbe-4436-b51d-15b5c2eeb0a8 req-02a79410-d9b3-4010-9b59-cd7d23016a14 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:56.551802 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b644436c-bbbe-4436-b51d-15b5c2eeb0a8 req-02a79410-d9b3-4010-9b59-cd7d23016a14 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0979753 May 31 11:57:56.604179 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6fc7a2c3-1a98-4fb3-b50a-bc702a2a6029 req-fe924efb-4536-4003-9d1c-ca9baa0bebc3 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:56.604953 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b644436c-bbbe-4436-b51d-15b5c2eeb0a8 req-92db7072-b9e5-403f-856e-10da4cb202fc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0453954 May 31 11:57:56.605656 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6fc7a2c3-1a98-4fb3-b50a-bc702a2a6029 req-fe924efb-4536-4003-9d1c-ca9baa0bebc3 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0976231 May 31 11:57:56.664401 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6fc7a2c3-1a98-4fb3-b50a-bc702a2a6029 req-15a788ad-0867-41a0-b8ec-d30f115a31bd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0528870 May 31 11:57:57.265308 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-142e435a-e472-45e5-b117-898cb76a9e34 req-f344a685-35b8-453c-9d50-592a1d93050b tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0439386 May 31 11:57:57.304132 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6ec1d06-79c8-4418-8567-7bf3c36e971d req-8265d6fa-176c-48ac-8fa6-f1d0ae786eb3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:57.305696 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a6ec1d06-79c8-4418-8567-7bf3c36e971d req-8265d6fa-176c-48ac-8fa6-f1d0ae786eb3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0841439 May 31 11:57:57.350056 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a6ec1d06-79c8-4418-8567-7bf3c36e971d req-8fcfaa41-b4e5-4afd-819b-9674cddbca44 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0387666 May 31 11:57:57.583529 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d93a1fa0-c398-49b5-a64c-2f3958ab930d req-6dda2062-032a-4ef9-a4f6-68986875409f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:57.584915 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d93a1fa0-c398-49b5-a64c-2f3958ab930d req-6dda2062-032a-4ef9-a4f6-68986875409f tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0868239 May 31 11:57:57.631286 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d93a1fa0-c398-49b5-a64c-2f3958ab930d req-8dac993c-de58-4ef8-81d5-d10da28eb354 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403512 May 31 11:57:57.687576 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b97785b6-1a17-4d09-8a71-7e233972041e req-5ac05c5e-92d0-4cc3-ab88-88b9d48f8a3b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:57.689094 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b97785b6-1a17-4d09-8a71-7e233972041e req-5ac05c5e-92d0-4cc3-ab88-88b9d48f8a3b service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=39ae37364a184ae7ae341ea7b2847c35&device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1137 time: 0.0747826 May 31 11:57:57.866320 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aaf4f343-60f0-4e8c-a7ef-6a59160a412e req-72d83909-ea72-4c96-be60-eb81a845fc02 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:57.867790 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-aaf4f343-60f0-4e8c-a7ef-6a59160a412e req-72d83909-ea72-4c96-be60-eb81a845fc02 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0805385 May 31 11:57:57.903183 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b97785b6-1a17-4d09-8a71-7e233972041e req-0dd0a006-cc4a-4da9-9c22-fee7b36213a6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:57.904629 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b97785b6-1a17-4d09-8a71-7e233972041e req-0dd0a006-cc4a-4da9-9c22-fee7b36213a6 service neutron] 10.222.0.47 "GET /v2.0/networks?id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb HTTP/1.1" status: 200 len: 915 time: 0.2011459 May 31 11:57:57.914707 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-aaf4f343-60f0-4e8c-a7ef-6a59160a412e req-281639ef-ec74-476f-8208-11b75f9039f0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0411239 May 31 11:57:57.916647 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-736b8288-854a-47c5-812d-974d0e012108 req-ab0160bc-471b-4395-8473-98dec92e09b8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:57.923632 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-736b8288-854a-47c5-812d-974d0e012108 req-ab0160bc-471b-4395-8473-98dec92e09b8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.1913764 May 31 11:57:58.047214 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-736b8288-854a-47c5-812d-974d0e012108 req-7b6dc534-5e9c-4b31-af36-083961a55c49 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.1226029 May 31 11:57:58.056260 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b97785b6-1a17-4d09-8a71-7e233972041e req-517910c5-2482-463b-acf5-bdc408cb6fbe service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 1001 time: 0.1178758 May 31 11:57:58.123708 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b97785b6-1a17-4d09-8a71-7e233972041e req-fa08e95c-76b2-40af-b07d-4e9efb2495b8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:58.125045 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b97785b6-1a17-4d09-8a71-7e233972041e req-fa08e95c-76b2-40af-b07d-4e9efb2495b8 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=672ec900-b55c-44a0-9ed9-7e27beeace12 HTTP/1.1" status: 200 len: 853 time: 0.0592346 May 31 11:57:58.204223 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b97785b6-1a17-4d09-8a71-7e233972041e req-8b38234b-52ca-475e-ba7c-9f90278d001d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:58.205699 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b97785b6-1a17-4d09-8a71-7e233972041e req-8b38234b-52ca-475e-ba7c-9f90278d001d service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0718594 May 31 11:57:58.422460 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b97785b6-1a17-4d09-8a71-7e233972041e req-619c4ad3-2357-459c-bd28-eea34e1199d0 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2011595 May 31 11:57:58.429561 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-748ad784-f81a-4fca-be87-26b0c9f336d3 req-46f83502-3370-437e-a318-9c24fb7d5ebb tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0526483 May 31 11:57:58.615854 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b97785b6-1a17-4d09-8a71-7e233972041e req-8db2595a-e2d0-4338-9304-af1fb8edc560 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1829574 May 31 11:57:58.631092 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbc5e7e1-67cf-4a8b-90c8-e231b406328d req-d3528082-ac11-4dab-a095-3b4e18d36948 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:58.632432 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dbc5e7e1-67cf-4a8b-90c8-e231b406328d req-d3528082-ac11-4dab-a095-3b4e18d36948 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.1294973 May 31 11:57:58.680924 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dbc5e7e1-67cf-4a8b-90c8-e231b406328d req-e0af9803-ac2a-4d2b-9f47-034f36a96a9f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422392 May 31 11:57:58.821975 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-787356bf-95fb-4ea5-8fd2-ae6302b46c65 req-415953f9-ae38-4c3b-9818-1eb4be0f9bba tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:58.823597 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-787356bf-95fb-4ea5-8fd2-ae6302b46c65 req-415953f9-ae38-4c3b-9818-1eb4be0f9bba tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.0806551 May 31 11:57:58.846598 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-e333b4c1-e443-47fc-8a0e-680ecfbd7055 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.47 "GET /v2.0/networks?tenant_id=d4b0a729336241e29d7d74dcbc6dc060&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0272624 May 31 11:57:58.868792 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-787356bf-95fb-4ea5-8fd2-ae6302b46c65 req-fe5e593c-bd10-4d02-9dfe-237ea7b3a96e tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0392747 May 31 11:57:59.016284 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-1aa55311-fee6-48b1-a783-7a6b25385de9 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:59.017732 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-1aa55311-fee6-48b1-a783-7a6b25385de9 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.1647096 May 31 11:57:59.034602 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Request body: {'port': {'device_id': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b', 'admin_state_up': True, 'tenant_id': 'd4b0a729336241e29d7d74dcbc6dc060'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:57:59.242509 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47740b00-d225-48f6-bad4-f6bea4216ec6 req-afd1cb09-86bd-43fd-ae1e-b40f5a3be24b tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:59.243979 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-47740b00-d225-48f6-bad4-f6bea4216ec6 req-afd1cb09-86bd-43fd-ae1e-b40f5a3be24b tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0849195 May 31 11:57:59.247561 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75bf78dd-de8b-45c2-87e0-3590950908e2 req-6cb0c23d-422e-4a75-8aeb-421d1d9158f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:59.249121 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-75bf78dd-de8b-45c2-87e0-3590950908e2 req-6cb0c23d-422e-4a75-8aeb-421d1d9158f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.2188473 May 31 11:57:59.250147 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:59.255130 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Usage tracker for resource:port and tenant:d4b0a729336241e29d7d74dcbc6dc060 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:59.259791 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:59.278606 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:59.292361 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-47740b00-d225-48f6-bad4-f6bea4216ec6 req-2db527c4-000a-45a2-861d-ea667b7c7bcc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0420203 May 31 11:57:59.317098 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-75bf78dd-de8b-45c2-87e0-3590950908e2 req-b53f67c0-579e-4222-9c45-739cc5a2b6cf tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0587935 May 31 11:57:59.390112 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Made reservation on behalf of d4b0a729336241e29d7d74dcbc6dc060 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:57:59.393809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:59.407093 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:59.422446 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:57:59.426809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Usage tracker for resource:security_group_rule and tenant:d4b0a729336241e29d7d74dcbc6dc060 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:57:59.429685 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:57:59.435359 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:57:59.531262 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:59.582043 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:57:59.586564 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3a819eb1-27fe-4725-a947-cf3adfa644b6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:57:59.590087 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1dac52ec-86de-42e0-bb3a-a236495b8fe3 req-505e615c-0544-46ef-a94b-3a51f7380328 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.0498555 May 31 11:57:59.662624 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Pushing event updated for resources: {'SecurityGroup': ['ID=c99cbc4e-0997-40d0-9734-767b70686311,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:57:59.671425 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:57:59.757593 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Allocated IP 192.168.233.24 (1b392bba-a31f-4f82-a1fa-40181dc52c6b/0baa2a0d-0223-4ead-bf59-0b788515e70d/cc7a1afb-a9a6-4702-81b7-99edecc48d2c) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:57:59.854284 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-df3c8d02-7c89-4138-bcb9-0a2fbd06bb04 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:57:59.855760 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-df3c8d02-7c89-4138-bcb9-0a2fbd06bb04 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.0850334 May 31 11:57:59.999182 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac39a957-50ed-4af5-bcc0-beae03b88731 req-bc5217c5-2abb-48d6-8652-d655643cc88e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:00.000668 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ac39a957-50ed-4af5-bcc0-beae03b88731 req-bc5217c5-2abb-48d6-8652-d655643cc88e tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 946 time: 0.2019031 May 31 11:58:00.012036 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Deleting port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:00.012396 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.064185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.087943 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ac39a957-50ed-4af5-bcc0-beae03b88731 req-815a95a2-d6ad-48c1-bd6b-d34d44f70c70 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0752552 May 31 11:58:00.090344 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Transition to ACTIVE for port object cc7a1afb-a9a6-4702-81b7-99edecc48d2c will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:58:00.184709 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.191325 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5f441a5d-e28d-4a33-a1ba-78131dc1a061 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:00.249943 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] For port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:00.250291 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.257107 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Network 1b392bba-a31f-4f82-a1fa-40181dc52c6b is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:58:00.260188 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:58:00.263903 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:58:00.269926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:00.270642 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.282573 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Pushing event updated for resources: {'Port': ['ID=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:00.294425 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:00.385154 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 937 time: 1.3559964 May 31 11:58:00.404174 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Request body: {'port': {'device_id': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:58:00.455374 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c7e90e6-9ffb-49f9-90d0-bc5ef65e68ab req-1047986d-45e6-4223-bfe2-a90fb0f0db80 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:00.457433 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1c7e90e6-9ffb-49f9-90d0-bc5ef65e68ab req-1047986d-45e6-4223-bfe2-a90fb0f0db80 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.4172914 May 31 11:58:00.466563 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Calling delete_port for 24ab9cdc-1297-4bcd-aea2-a477b3dd537c owned by compute:nova {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:00.553124 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1c7e90e6-9ffb-49f9-90d0-bc5ef65e68ab req-fa3482c4-66fe-4e23-a82f-5b60144c7d60 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0791173 May 31 11:58:00.579398 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.616988 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-3f3d0a75-f4f5-4bcc-8194-fb14a59a8e68 None None] Provisioning for port cc7a1afb-a9a6-4702-81b7-99edecc48d2c completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:58:00.622774 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-3f3d0a75-f4f5-4bcc-8194-fb14a59a8e68 None None] Provisioning complete for port cc7a1afb-a9a6-4702-81b7-99edecc48d2c triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:58:00.623160 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3f3d0a75-f4f5-4bcc-8194-fb14a59a8e68 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.651038 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b1351941-c46d-4360-a709-c07f4018f74d req-8a6bfeff-4037-4f07-88bb-e52e903a91b4 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:00.659966 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b1351941-c46d-4360-a709-c07f4018f74d req-8a6bfeff-4037-4f07-88bb-e52e903a91b4 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.2235556 May 31 11:58:00.684427 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-3f3d0a75-f4f5-4bcc-8194-fb14a59a8e68 None None] Port cc7a1afb-a9a6-4702-81b7-99edecc48d2c cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:58:00.747013 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.755408 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:00.759100 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['c4a233f2-9446-42a7-964d-1dc2496c938a', '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8', '150f3a3b-28b5-435a-b826-0eee61f6b143'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:58:00.764788 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b1351941-c46d-4360-a709-c07f4018f74d req-a9ed7cbe-3ffe-43a3-ac41-74f30a00db9c tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.1037455 May 31 11:58:00.907704 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Pushing event deleted for resources: {'Port': ['ID=24ab9cdc-1297-4bcd-aea2-a477b3dd537c,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:00.923611 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.168s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:00.928591 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] No DVR routers for this DVR port 24ab9cdc-1297-4bcd-aea2-a477b3dd537c on host hv-cinder-79123 {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:58:00.936402 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:00.938891 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '1c76757c-f367-4ba4-8e44-150f317f153e', 'name': 'network-vif-deleted', 'tag': '24ab9cdc-1297-4bcd-aea2-a477b3dd537c'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:00.941996 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-7fb67292-eca8-4a76-9e61-c050fb34ee9d" -d '{"events": [{"server_uuid": "1c76757c-f367-4ba4-8e44-150f317f153e", "name": "network-vif-deleted", "tag": "24ab9cdc-1297-4bcd-aea2-a477b3dd537c"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:00.969072 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-bbe1a327-159d-4fd9-a2a2-8781e720e6dd tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:00.970514 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-bbe1a327-159d-4fd9-a2a2-8781e720e6dd tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.47 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 945 time: 0.3122742 May 31 11:58:00.985681 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-694e5594-ef76-4491-b9d9-b053886c14d4 req-be9a5911-22f9-40ef-b72c-79d0dc3b6b24 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:00.987302 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-694e5594-ef76-4491-b9d9-b053886c14d4 req-be9a5911-22f9-40ef-b72c-79d0dc3b6b24 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.3308232 May 31 11:58:01.091174 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f6d6448-3ad6-4c7a-94ed-dd48efa7624f req-52921fe3-0a66-43d3-995e-756dda17a6f9 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:01.091901 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-9f6d6448-3ad6-4c7a-94ed-dd48efa7624f req-52921fe3-0a66-43d3-995e-756dda17a6f9 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Synchronizing usage tracker for tenant:d4b0a729336241e29d7d74dcbc6dc060 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:58:01.119352 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "DELETE /v2.0/ports/24ab9cdc-1297-4bcd-aea2-a477b3dd537c HTTP/1.1" status: 204 len: 173 time: 1.2564888 May 31 11:58:01.122044 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-694e5594-ef76-4491-b9d9-b053886c14d4 req-f3b5de65-3485-432b-97f1-519eed136af9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0953467 May 31 11:58:01.184344 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Deleting port 3670cfae-0d92-4d02-a5ba-5c373efc785e {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:01.184705 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.201199 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id cc7a1afb-a9a6-4702-81b7-99edecc48d2c {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:58:01.211088 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.229377 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Transition to ACTIVE for port object cc7a1afb-a9a6-4702-81b7-99edecc48d2c will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:58:01.260320 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:00 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-a81e876d-befe-49a6-83fe-231a21c11dc0 x-openstack-request-id: req-a81e876d-befe-49a6-83fe-231a21c11dc0 {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:01.260631 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "1c76757c-f367-4ba4-8e44-150f317f153e", "name": "network-vif-deleted", "tag": "24ab9cdc-1297-4bcd-aea2-a477b3dd537c", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:01.260904 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-a81e876d-befe-49a6-83fe-231a21c11dc0 {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:01.261310 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '1c76757c-f367-4ba4-8e44-150f317f153e', 'name': 'network-vif-deleted', 'tag': '24ab9cdc-1297-4bcd-aea2-a477b3dd537c', 'status': 'completed', 'code': 200} May 31 11:58:01.264769 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-9f6d6448-3ad6-4c7a-94ed-dd48efa7624f req-52921fe3-0a66-43d3-995e-756dda17a6f9 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Unset dirty status for tenant:d4b0a729336241e29d7d74dcbc6dc060 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:58:01.266165 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9f6d6448-3ad6-4c7a-94ed-dd48efa7624f req-52921fe3-0a66-43d3-995e-756dda17a6f9 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 935 time: 0.3778710 May 31 11:58:01.316530 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-9f6d6448-3ad6-4c7a-94ed-dd48efa7624f req-e5cc57c8-8f61-4ab9-99f5-f1b4192bb312 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Synchronizing usage tracker for tenant:d4b0a729336241e29d7d74dcbc6dc060 on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:58:01.379686 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-b8453dd1-e117-487e-ac64-b40e7c20a7f6 req-959e05c8-a717-4581-8a6f-55aece5e4f1d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:58:01.383044 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] For port 3670cfae-0d92-4d02-a5ba-5c373efc785e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3670cfae-0d92-4d02-a5ba-5c373efc785e,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:01.383426 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.404611 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.409097 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:01.529401 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-9f6d6448-3ad6-4c7a-94ed-dd48efa7624f req-e5cc57c8-8f61-4ab9-99f5-f1b4192bb312 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Unset dirty status for tenant:d4b0a729336241e29d7d74dcbc6dc060 on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:58:01.530638 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9f6d6448-3ad6-4c7a-94ed-dd48efa7624f req-e5cc57c8-8f61-4ab9-99f5-f1b4192bb312 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2584386 May 31 11:58:01.539134 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Calling delete_port for 3670cfae-0d92-4d02-a5ba-5c373efc785e owned by compute:nova {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:01.578347 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Pushing event updated for resources: {'Port': ['ID=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:01.588313 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:01.612404 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Attempting to bind port cc7a1afb-a9a6-4702-81b7-99edecc48d2c on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:58:01.612871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Attempting to bind port cc7a1afb-a9a6-4702-81b7-99edecc48d2c by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:58:01.613237 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Attempting to bind port cc7a1afb-a9a6-4702-81b7-99edecc48d2c on network 1b392bba-a31f-4f82-a1fa-40181dc52c6b {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:58:01.620414 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Port cc7a1afb-a9a6-4702-81b7-99edecc48d2c on network 1b392bba-a31f-4f82-a1fa-40181dc52c6b not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:58:01.620968 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Attempting to bind port cc7a1afb-a9a6-4702-81b7-99edecc48d2c on network 1b392bba-a31f-4f82-a1fa-40181dc52c6b {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:58:01.630108 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 57, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:58:01.630779 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Checking segment: {'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:58:01.631489 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Bound using segment: {'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:58:01.632155 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Bound port: cc7a1afb-a9a6-4702-81b7-99edecc48d2c, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'a7256a82-57c5-4819-8aea-511768fbafdc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 358, 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:58:01.640540 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-b8453dd1-e117-487e-ac64-b40e7c20a7f6 req-959e05c8-a717-4581-8a6f-55aece5e4f1d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:58:01.642048 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b8453dd1-e117-487e-ac64-b40e7c20a7f6 req-959e05c8-a717-4581-8a6f-55aece5e4f1d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=1c76757c-f367-4ba4-8e44-150f317f153e HTTP/1.1" status: 200 len: 210 time: 0.3275223 May 31 11:58:01.646323 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.770789 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.777005 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-db6d0d33-efbf-43ec-91bc-76e946a2f56f service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:01.816618 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] For port cc7a1afb-a9a6-4702-81b7-99edecc48d2c, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:58:01.822289 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] For port cc7a1afb-a9a6-4702-81b7-99edecc48d2c, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,segment=NetworkSegment(a7256a82-57c5-4819-8aea-511768fbafdc),segment_id=a7256a82-57c5-4819-8aea-511768fbafdc)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:58:01.825561 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Pushing event deleted for resources: {'Port': ['ID=3670cfae-0d92-4d02-a5ba-5c373efc785e,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:01.830390 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:01.849978 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0135e39f-e0f9-4468-af79-20c8058ee37b req-58d3b818-09dd-4562-97c9-893677a2a0a7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:01.851586 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0135e39f-e0f9-4468-af79-20c8058ee37b req-58d3b818-09dd-4562-97c9-893677a2a0a7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 867 time: 0.1407022 May 31 11:58:01.900999 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0135e39f-e0f9-4468-af79-20c8058ee37b req-d9ecf2e1-586e-46f5-9adb-1e2c449a4bbd tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430951 May 31 11:58:01.924707 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.928830 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:01.949100 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['f18f632c-d53b-4423-93a7-f93c3e8e0778', '3670cfae-0d92-4d02-a5ba-5c373efc785e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:58:01.950757 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] No DVR routers for this DVR port 3670cfae-0d92-4d02-a5ba-5c373efc785e on host hv-cinder-79123 {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:58:01.954434 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.987344 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:01.988003 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:01.989377 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] 10.222.0.47 "PUT /v2.0/ports/cc7a1afb-a9a6-4702-81b7-99edecc48d2c HTTP/1.1" status: 200 len: 1097 time: 1.5900507 May 31 11:58:01.989886 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'tag': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:01.992773 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-9642fa97-77e4-4942-8ab3-7fb03bb3b17f" -d '{"events": [{"name": "network-changed", "server_uuid": "d7675d4e-5c62-4304-b0fc-603206d7a8e6", "tag": "cc7a1afb-a9a6-4702-81b7-99edecc48d2c"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:02.009141 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18723d90-38ca-4e52-a860-959d1fc6be56 req-5fed7389-924e-486c-952a-6d0349b30d24 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:02.010981 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-18723d90-38ca-4e52-a860-959d1fc6be56 req-5fed7389-924e-486c-952a-6d0349b30d24 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.1177382 May 31 11:58:02.024466 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Pushing event updated for resources: {'Port': ['ID=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:02.027603 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:02.057428 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:58:01 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-72c6951b-f05f-4c6f-991f-b41097d218a9 x-openstack-request-id: req-72c6951b-f05f-4c6f-991f-b41097d218a9 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:02.057775 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "d7675d4e-5c62-4304-b0fc-603206d7a8e6", "tag": "cc7a1afb-a9a6-4702-81b7-99edecc48d2c", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:02.058062 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-72c6951b-f05f-4c6f-991f-b41097d218a9 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:02.058489 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'tag': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c', 'status': 'completed', 'code': 200} May 31 11:58:02.111071 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-18723d90-38ca-4e52-a860-959d1fc6be56 req-73726e0e-d9b6-45bd-827e-1877530f82af tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0927045 May 31 11:58:02.118058 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2b3f10f9-2e76-4adc-b077-04e246d12a47 req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.47 "DELETE /v2.0/ports/3670cfae-0d92-4d02-a5ba-5c373efc785e HTTP/1.1" status: 204 len: 173 time: 1.1282048 May 31 11:58:02.174211 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9642fa97-77e4-4942-8ab3-7fb03bb3b17f req-2cf202db-b17b-4f9f-ab95-3d0bf59c0525 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:02.176280 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9642fa97-77e4-4942-8ab3-7fb03bb3b17f req-2cf202db-b17b-4f9f-ab95-3d0bf59c0525 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=d4b0a729336241e29d7d74dcbc6dc060&device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 1100 time: 0.0946412 May 31 11:58:02.216529 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9642fa97-77e4-4942-8ab3-7fb03bb3b17f req-59a6fc0b-be28-49ba-9bfd-44f7ded8fe8b service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=d4b0a729336241e29d7d74dcbc6dc060&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0317557 May 31 11:58:02.404694 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24ab9cdc-1297-4bcd-aea2-a477b3dd537c no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:02.419118 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9642fa97-77e4-4942-8ab3-7fb03bb3b17f req-4191afea-c8d0-4b3c-ad99-9525b4289f1e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:02.420714 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9642fa97-77e4-4942-8ab3-7fb03bb3b17f req-4191afea-c8d0-4b3c-ad99-9525b4289f1e service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1971104 May 31 11:58:02.439816 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3067c02d-998c-49c6-8859-375e19649e64 req-ec0b87b7-f7f8-49ec-b2fb-78003647c198 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:02.442517 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3067c02d-998c-49c6-8859-375e19649e64 req-ec0b87b7-f7f8-49ec-b2fb-78003647c198 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.1436894 May 31 11:58:02.443188 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 24ab9cdc-1297-4bcd-aea2-a477b3dd537c {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:58:02.443537 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24ab9cdc-1297-4bcd-aea2-a477b3dd537c not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:58:02.443958 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 3670cfae-0d92-4d02-a5ba-5c373efc785e no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:02.481383 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 3670cfae-0d92-4d02-a5ba-5c373efc785e {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:58:02.481755 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 3670cfae-0d92-4d02-a5ba-5c373efc785e not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:58:02.532017 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3067c02d-998c-49c6-8859-375e19649e64 req-c03fb0dd-291a-4f93-bb1a-bfd1662ea12a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0826621 May 31 11:58:02.571305 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-9c4bd059-fc0a-4d7e-8cfb-ed5816abee26 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:02.573036 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-9c4bd059-fc0a-4d7e-8cfb-ed5816abee26 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=d4b0a729336241e29d7d74dcbc6dc060&device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 1100 time: 0.0937831 May 31 11:58:02.648473 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-797aeb50-af05-4805-87e4-1463f5adb5f0 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.24&port_id=cc7a1afb-a9a6-4702-81b7-99edecc48d2c HTTP/1.1" status: 200 len: 217 time: 0.0656610 May 31 11:58:02.720056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-5e846c5f-8435-4f9f-b236-802e4651538f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:02.721518 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-5e846c5f-8435-4f9f-b236-802e4651538f service neutron] 10.222.0.47 "GET /v2.0/subnets?id=0baa2a0d-0223-4ead-bf59-0b788515e70d HTTP/1.1" status: 200 len: 843 time: 0.0658493 May 31 11:58:02.730396 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1a35b00-e48a-440d-8e9e-6d2870e5a890 req-fc61f1c4-e17d-40f7-9cc2-cc5da0cc4d7f tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:02.731921 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e1a35b00-e48a-440d-8e9e-6d2870e5a890 req-fc61f1c4-e17d-40f7-9cc2-cc5da0cc4d7f tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 947 time: 0.0846896 May 31 11:58:02.781202 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e1a35b00-e48a-440d-8e9e-6d2870e5a890 req-d8b42ca6-b37a-4c5c-ba17-c90f72cdb1e9 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0423536 May 31 11:58:02.807274 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-2442535f-9cff-4c82-b1b2-ce04b3b6164b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:02.808726 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-2442535f-9cff-4c82-b1b2-ce04b3b6164b service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=1b392bba-a31f-4f82-a1fa-40181dc52c6b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.0784585 May 31 11:58:02.945245 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-f49e89fc-4f12-473e-836d-dab060c0d444 service neutron] 10.222.0.47 "GET /v2.0/networks/1b392bba-a31f-4f82-a1fa-40181dc52c6b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1246188 May 31 11:58:03.168650 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-62c8606f-4621-4cfa-8c77-173c205a7b6b req-6a562c71-49d9-4e12-9e38-f4b2d5c9b024 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=71751849-4dff-4706-b4fe-d196e7c2e6e2 HTTP/1.1" status: 200 len: 210 time: 0.0622191 May 31 11:58:03.221626 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c72aea73-38d8-4a30-8b3d-c2094fe3f5c6 req-86688072-a5da-4f43-aba2-3a79bd04bc84 service neutron] 10.222.0.47 "GET /v2.0/networks/1b392bba-a31f-4f82-a1fa-40181dc52c6b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2634904 May 31 11:58:03.263674 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '71751849-4dff-4706-b4fe-d196e7c2e6e2', 'name': 'network-vif-deleted', 'tag': '3670cfae-0d92-4d02-a5ba-5c373efc785e'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:03.266096 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-cec60f1c-81f5-4e66-8eaa-180bff007830" -d '{"events": [{"server_uuid": "71751849-4dff-4706-b4fe-d196e7c2e6e2", "name": "network-vif-deleted", "tag": "3670cfae-0d92-4d02-a5ba-5c373efc785e"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:03.310687 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:03 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-ed5b17b5-cffb-4a64-af9d-b08d05a7058b x-openstack-request-id: req-ed5b17b5-cffb-4a64-af9d-b08d05a7058b {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:03.311002 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "71751849-4dff-4706-b4fe-d196e7c2e6e2", "name": "network-vif-deleted", "tag": "3670cfae-0d92-4d02-a5ba-5c373efc785e", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:03.311270 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-ed5b17b5-cffb-4a64-af9d-b08d05a7058b {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:03.311686 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '71751849-4dff-4706-b4fe-d196e7c2e6e2', 'name': 'network-vif-deleted', 'tag': '3670cfae-0d92-4d02-a5ba-5c373efc785e', 'status': 'completed', 'code': 200} May 31 11:58:03.323209 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eaba5c40-cc6c-4a9d-8158-b1c11255b66f req-1e1bb4a1-f705-468d-bcc9-bf4e77012852 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:03.324675 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-eaba5c40-cc6c-4a9d-8158-b1c11255b66f req-1e1bb4a1-f705-468d-bcc9-bf4e77012852 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0958846 May 31 11:58:03.371134 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-eaba5c40-cc6c-4a9d-8158-b1c11255b66f req-78830ce2-da71-42b3-9496-8e2c36e2e366 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0405622 May 31 11:58:03.726912 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-587b5512-22fe-4456-bb6b-0adf3b5b5b05 req-05aa9a8d-4eb6-4b01-a5d0-88eb2e2fd1a7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:03.728343 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-587b5512-22fe-4456-bb6b-0adf3b5b5b05 req-05aa9a8d-4eb6-4b01-a5d0-88eb2e2fd1a7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 943 time: 0.0791786 May 31 11:58:03.771779 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-587b5512-22fe-4456-bb6b-0adf3b5b5b05 req-5945cf4c-6fb9-4c63-ae96-56a6b67e9c2d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0376413 May 31 11:58:04.045374 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10827f13-a9fc-4fea-b850-167635927ee0 req-54a1cd1c-1db2-4d2d-a7ad-c334bd272b75 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:04.046766 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-10827f13-a9fc-4fea-b850-167635927ee0 req-54a1cd1c-1db2-4d2d-a7ad-c334bd272b75 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 947 time: 0.1554422 May 31 11:58:04.091461 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-10827f13-a9fc-4fea-b850-167635927ee0 req-469cb871-bb5a-42b1-a82f-a542dd295869 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0386155 May 31 11:58:04.418469 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:58:04.562496 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:04.575860 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'port_id': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'mac_address': 'fa:16:3e:5a:6f:46', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 371, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:58:04.630343 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:04.648387 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id e0a8cb19-42fc-4a14-b86a-efda3d732b98 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:58:04.665848 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:04.776069 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:04.778042 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:04.852027 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=9']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:04.859249 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.859658 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.860053 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.860445 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.860827 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.861211 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.861618 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.862004 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.862442 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.863319 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.863693 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.864073 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:04.867964 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:04.915662 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbd0f864-360a-4a77-a3a4-eca1896d851d req-423db6b3-ba56-43b2-a6d7-c5aec4fe7d6f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:04.917301 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fbd0f864-360a-4a77-a3a4-eca1896d851d req-423db6b3-ba56-43b2-a6d7-c5aec4fe7d6f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0862851 May 31 11:58:04.950571 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0511e9c-6532-4f1c-b43c-77b30b26fff2 req-02ffe05b-b6fe-4b4c-9a76-931fac927cdf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:04.952020 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0511e9c-6532-4f1c-b43c-77b30b26fff2 req-02ffe05b-b6fe-4b4c-9a76-931fac927cdf tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0819898 May 31 11:58:04.968269 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fbd0f864-360a-4a77-a3a4-eca1896d851d req-45a195d4-6b97-4cac-ae41-10eeaa0a682c tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0450928 May 31 11:58:04.998554 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0511e9c-6532-4f1c-b43c-77b30b26fff2 req-9dab3331-30cb-46e6-9fd6-11d3ee3d83d7 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403845 May 31 11:58:05.085011 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['e0a8cb19-42fc-4a14-b86a-efda3d732b98'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:58:05.163231 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c4b751a8-5513-489d-a0d8-e704fb39e7f7 req-73d4d640-4f9f-4a58-a636-db0dca67546b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:05.165434 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c4b751a8-5513-489d-a0d8-e704fb39e7f7 req-73d4d640-4f9f-4a58-a636-db0dca67546b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 946 time: 0.0978308 May 31 11:58:05.174281 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8eb2eefa-48b1-437e-8b28-ffde987e5be4 req-41618157-b407-4e59-a397-9ff710862933 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:05.176327 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8eb2eefa-48b1-437e-8b28-ffde987e5be4 req-41618157-b407-4e59-a397-9ff710862933 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 944 time: 0.0847356 May 31 11:58:05.214283 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c4b751a8-5513-489d-a0d8-e704fb39e7f7 req-a2d14076-1ef9-41b5-b44b-7a6b08e0f89f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416932 May 31 11:58:05.227069 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8eb2eefa-48b1-437e-8b28-ffde987e5be4 req-1833ef79-a116-4c5d-86fb-5c1d2af396d4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0445061 May 31 11:58:05.390578 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-205b28d6-1c51-4241-b224-098ecdb1673a req-d4c79564-0ad0-4727-9155-f21ee56f033a tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:05.391978 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-205b28d6-1c51-4241-b224-098ecdb1673a req-d4c79564-0ad0-4727-9155-f21ee56f033a tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 947 time: 0.0804014 May 31 11:58:05.436003 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-205b28d6-1c51-4241-b224-098ecdb1673a req-59acc2d0-11aa-4e7e-8a3a-bbc5575b2dd7 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0371585 May 31 11:58:06.161448 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-319ace36-c53c-42e4-b3e8-780a0e3c6f2b req-3190b2ae-874d-4fea-b013-575a63cbf42d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:06.164280 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-319ace36-c53c-42e4-b3e8-780a0e3c6f2b req-3190b2ae-874d-4fea-b013-575a63cbf42d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0853305 May 31 11:58:06.187183 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8305c8c3-423e-4520-8325-df93052b7fe4 req-a8bb316d-b407-46b0-9d1f-dd5ed70d09f8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:06.187972 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-8305c8c3-423e-4520-8325-df93052b7fe4 req-a8bb316d-b407-46b0-9d1f-dd5ed70d09f8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Synchronizing usage tracker for tenant:c94978b38250481095d2824adfb16ed4 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:58:06.212532 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-319ace36-c53c-42e4-b3e8-780a0e3c6f2b req-49b5ca8a-38d8-46cd-8f49-542ae7fa5cfc tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0427957 May 31 11:58:06.287237 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-8305c8c3-423e-4520-8325-df93052b7fe4 req-a8bb316d-b407-46b0-9d1f-dd5ed70d09f8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Unset dirty status for tenant:c94978b38250481095d2824adfb16ed4 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:58:06.288444 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8305c8c3-423e-4520-8325-df93052b7fe4 req-a8bb316d-b407-46b0-9d1f-dd5ed70d09f8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1838307 May 31 11:58:06.336149 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8305c8c3-423e-4520-8325-df93052b7fe4 req-2c7ed9dd-0c84-4573-971b-b01330acd48a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0418618 May 31 11:58:06.411435 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2731970f-196e-4e31-b481-250a578c8323 req-b3821310-989a-4950-8596-1a1f3e0ae981 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:06.412869 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2731970f-196e-4e31-b481-250a578c8323 req-b3821310-989a-4950-8596-1a1f3e0ae981 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 946 time: 0.0879345 May 31 11:58:06.427251 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device cc7a1afb-a9a6-4702-81b7-99edecc48d2c details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:58:06.433711 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:58:06.461681 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2731970f-196e-4e31-b481-250a578c8323 req-10f6d4f4-9f4e-42fd-a1b8-8b11c2e38f66 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0429466 May 31 11:58:06.535307 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d01fec7-7f63-4d6e-9851-e5b1c84a9a9e req-ccbb554a-ca2d-4842-9e3d-fdc9264bdaf8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:06.536989 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0d01fec7-7f63-4d6e-9851-e5b1c84a9a9e req-ccbb554a-ca2d-4842-9e3d-fdc9264bdaf8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 944 time: 0.0993295 May 31 11:58:06.587407 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0d01fec7-7f63-4d6e-9851-e5b1c84a9a9e req-794a6166-dbd1-457f-8f76-3315ca215535 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424225 May 31 11:58:06.634491 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port e0a8cb19-42fc-4a14-b86a-efda3d732b98 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:58:06.634870 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:06.682939 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-fad1465a-66da-4a7d-a347-26674f896ff8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:06.684353 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-fad1465a-66da-4a7d-a347-26674f896ff8 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 946 time: 0.1500249 May 31 11:58:06.696628 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52c6259a-acb1-4661-a611-b5fa291e611e req-667c161f-0237-43df-8122-603423c9adbe tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:06.698010 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-52c6259a-acb1-4661-a611-b5fa291e611e req-667c161f-0237-43df-8122-603423c9adbe tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 947 time: 0.1216013 May 31 11:58:06.720136 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port cc7a1afb-a9a6-4702-81b7-99edecc48d2c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,segment=NetworkSegment(a7256a82-57c5-4819-8aea-511768fbafdc),segment_id=a7256a82-57c5-4819-8aea-511768fbafdc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:06.737236 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c', 'network_id': '1b392bba-a31f-4f82-a1fa-40181dc52c6b', 'port_id': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c', 'mac_address': 'fa:16:3e:4d:2a:a3', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 358, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '0baa2a0d-0223-4ead-bf59-0b788515e70d', 'ip_address': '192.168.233.24'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:58:06.825293 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-52c6259a-acb1-4661-a611-b5fa291e611e req-9d1094c2-45ed-4ac9-85e7-e25dde32b686 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0978456 May 31 11:58:06.843647 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:06.862106 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:06.871911 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Deleting port 150f3a3b-28b5-435a-b826-0eee61f6b143 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:06.872250 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:06.881906 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id cc7a1afb-a9a6-4702-81b7-99edecc48d2c {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:58:06.905700 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:06.913048 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-7830ce3d-bf95-49f2-b3df-f85642f8cf06" -d '{"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-plugged", "status": "completed", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:06.915420 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port cc7a1afb-a9a6-4702-81b7-99edecc48d2c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,segment=NetworkSegment(a7256a82-57c5-4819-8aea-511768fbafdc),segment_id=a7256a82-57c5-4819-8aea-511768fbafdc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:06.918962 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:06.958008 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:06 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-58d7552e-2693-4943-b52d-b14271207c05 x-openstack-request-id: req-58d7552e-2693-4943-b52d-b14271207c05 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:06.958308 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-plugged", "status": "completed", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:06.958566 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-58d7552e-2693-4943-b52d-b14271207c05 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:06.958961 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'code': 200} May 31 11:58:06.990216 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:06.992280 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5d0d9436-e352-4dfc-861c-5722119e5ef9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:07.013263 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] For port 150f3a3b-28b5-435a-b826-0eee61f6b143, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=150f3a3b-28b5-435a-b826-0eee61f6b143,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:07.013651 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:07.026702 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:07.059833 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:07.062771 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:07.063224 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.029s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:07.131202 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=10']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:07.134151 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:07.138595 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Calling delete_port for 150f3a3b-28b5-435a-b826-0eee61f6b143 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:07.309130 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['cc7a1afb-a9a6-4702-81b7-99edecc48d2c'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:58:07.412974 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a0707a8-2481-4e98-91ce-df4e751604ff req-7b2791f2-184c-4b62-b43a-9cd3c32dfe45 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:07.414506 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0a0707a8-2481-4e98-91ce-df4e751604ff req-7b2791f2-184c-4b62-b43a-9cd3c32dfe45 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0886593 May 31 11:58:07.437954 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:07.447232 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:07.519165 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Pushing event deleted for resources: {'Port': ['ID=150f3a3b-28b5-435a-b826-0eee61f6b143,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:07.524995 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:07.537090 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a68dbbe6-530c-43f3-95d9-8aeff2d1103d req-66062001-15ba-48d7-bee0-39bc220dd8c1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:07.539137 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a68dbbe6-530c-43f3-95d9-8aeff2d1103d req-66062001-15ba-48d7-bee0-39bc220dd8c1 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0881784 May 31 11:58:07.564766 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0a0707a8-2481-4e98-91ce-df4e751604ff req-a4300986-6f09-4fc4-980f-967af1cdb731 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.1421359 May 31 11:58:07.592714 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a68dbbe6-530c-43f3-95d9-8aeff2d1103d req-f57a10de-b368-45f4-ad0d-9140d6a72921 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0470381 May 31 11:58:07.803259 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6e3aa48c-90ac-4192-b4d2-6cc139fc4346 req-a0cc8fa9-a3a9-4885-a247-90c671e732b6 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 210 time: 0.1351373 May 31 11:58:07.851431 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7aece41-f566-4d52-b389-5289a204ff51 req-7f8d387d-9c09-4969-9e2c-974eecea5929 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:07.852941 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b7aece41-f566-4d52-b389-5289a204ff51 req-7f8d387d-9c09-4969-9e2c-974eecea5929 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 946 time: 0.1446035 May 31 11:58:07.859020 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08c23f34-1184-4cc9-89ec-8646403d83af req-da7e07a5-98ce-4b21-b309-cae4bbcd41c8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:07.860385 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-08c23f34-1184-4cc9-89ec-8646403d83af req-da7e07a5-98ce-4b21-b309-cae4bbcd41c8 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 1099 time: 0.1183910 May 31 11:58:07.929396 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b7aece41-f566-4d52-b389-5289a204ff51 req-c3d6f81c-f982-49dd-81ad-8258ced9d171 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0680389 May 31 11:58:07.932884 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-08c23f34-1184-4cc9-89ec-8646403d83af req-abbd59e5-cf56-454c-b580-67888dc0c197 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0566518 May 31 11:58:08.042047 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] No DVR routers for this DVR port 150f3a3b-28b5-435a-b826-0eee61f6b143 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:58:08.047925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:08.051488 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '231d5bcd-ca61-435d-8029-4d0537a194ad', 'name': 'network-vif-deleted', 'tag': '150f3a3b-28b5-435a-b826-0eee61f6b143'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:08.053903 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-9a5810dd-6202-46bf-aecf-e390742cb34d" -d '{"events": [{"server_uuid": "231d5bcd-ca61-435d-8029-4d0537a194ad", "name": "network-vif-deleted", "tag": "150f3a3b-28b5-435a-b826-0eee61f6b143"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:08.070297 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['c4a233f2-9446-42a7-964d-1dc2496c938a', '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8', '150f3a3b-28b5-435a-b826-0eee61f6b143'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:58:08.079784 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6fab056e-588c-4386-a3aa-663e4d307f26 req-8801c3dd-4d0a-42de-8921-a33359191e44 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:08.082202 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6fab056e-588c-4386-a3aa-663e4d307f26 req-8801c3dd-4d0a-42de-8921-a33359191e44 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 948 time: 0.1371891 May 31 11:58:08.104168 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:08 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-bfb6fa6a-c539-4fc2-b8bd-c7bb1ed12b36 x-openstack-request-id: req-bfb6fa6a-c539-4fc2-b8bd-c7bb1ed12b36 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:08.104484 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "231d5bcd-ca61-435d-8029-4d0537a194ad", "name": "network-vif-deleted", "tag": "150f3a3b-28b5-435a-b826-0eee61f6b143", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:08.104789 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-bfb6fa6a-c539-4fc2-b8bd-c7bb1ed12b36 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:08.105197 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '231d5bcd-ca61-435d-8029-4d0537a194ad', 'name': 'network-vif-deleted', 'tag': '150f3a3b-28b5-435a-b826-0eee61f6b143', 'status': 'completed', 'code': 200} May 31 11:58:08.134100 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6fab056e-588c-4386-a3aa-663e4d307f26 req-52598c8f-0227-4265-a59d-959279ab0363 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0453961 May 31 11:58:08.177931 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c324e606-a4f0-42da-8935-fb168ab9d8ab req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "DELETE /v2.0/ports/150f3a3b-28b5-435a-b826-0eee61f6b143 HTTP/1.1" status: 204 len: 173 time: 1.4788325 May 31 11:58:08.436793 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device cc7a1afb-a9a6-4702-81b7-99edecc48d2c up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:58:08.581496 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port cc7a1afb-a9a6-4702-81b7-99edecc48d2c completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:58:08.586719 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port cc7a1afb-a9a6-4702-81b7-99edecc48d2c triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:58:08.587131 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:08.727022 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:08.759816 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08c63eb5-1b23-4849-ba9a-3bfa770bbc8f req-85b26ef4-8338-40a3-8493-b52d0c2cf88e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:08.761029 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port cc7a1afb-a9a6-4702-81b7-99edecc48d2c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,segment=NetworkSegment(a7256a82-57c5-4819-8aea-511768fbafdc),segment_id=a7256a82-57c5-4819-8aea-511768fbafdc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:08.761268 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-08c63eb5-1b23-4849-ba9a-3bfa770bbc8f req-85b26ef4-8338-40a3-8493-b52d0c2cf88e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0853231 May 31 11:58:08.793906 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aade2377-0408-4292-91cd-cf0743358cc2 req-a5ab05e2-f652-473e-b791-f5c9d4b77522 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:08.795354 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-aade2377-0408-4292-91cd-cf0743358cc2 req-a5ab05e2-f652-473e-b791-f5c9d4b77522 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0836096 May 31 11:58:08.813969 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-08c63eb5-1b23-4849-ba9a-3bfa770bbc8f req-d51fb2b4-7a38-4c19-8cfc-9e8ef829b56e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0456975 May 31 11:58:08.828457 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:08.830366 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:08.846323 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-aade2377-0408-4292-91cd-cf0743358cc2 req-aade9eaf-45ae-4d64-b26d-2cf4885b8141 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0441532 May 31 11:58:08.909810 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:08.918396 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:08.960270 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:08.963011 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-548b64ad-e8b0-432e-ba97-2ca9df7098f8" -d '{"events": [{"server_uuid": "d7675d4e-5c62-4304-b0fc-603206d7a8e6", "name": "network-vif-plugged", "status": "completed", "tag": "cc7a1afb-a9a6-4702-81b7-99edecc48d2c"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:08.975962 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:08.992405 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5df643fd-6d67-4e92-aa3a-20edf8baa1fb req-a51a01b1-0fef-404d-b68c-94d4dff652fa tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=231d5bcd-ca61-435d-8029-4d0537a194ad HTTP/1.1" status: 200 len: 210 time: 0.0525789 May 31 11:58:09.050956 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:08 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-33d360d0-7229-46a1-9b62-169dc78737f0 x-openstack-request-id: req-33d360d0-7229-46a1-9b62-169dc78737f0 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:09.051451 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "d7675d4e-5c62-4304-b0fc-603206d7a8e6", "name": "network-vif-plugged", "status": "completed", "tag": "cc7a1afb-a9a6-4702-81b7-99edecc48d2c", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:09.051727 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-33d360d0-7229-46a1-9b62-169dc78737f0 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:09.052124 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c', 'code': 200} May 31 11:58:09.140693 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-797025e4-6c48-4c96-8fd6-bbd72304d6cf req-e133e5ad-ae09-4ee1-b8d5-5dd242cbf014 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:09.142151 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-797025e4-6c48-4c96-8fd6-bbd72304d6cf req-e133e5ad-ae09-4ee1-b8d5-5dd242cbf014 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 946 time: 0.0905504 May 31 11:58:09.195997 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-797025e4-6c48-4c96-8fd6-bbd72304d6cf req-7f3c0272-7ed9-48b2-be70-33078e81a6e2 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0477350 May 31 11:58:09.376855 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-159d6e35-dd5d-4996-b0d1-40d5c4dd645b req-d3f64a66-9bc6-4d27-a428-6b286c339961 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:09.378483 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-159d6e35-dd5d-4996-b0d1-40d5c4dd645b req-d3f64a66-9bc6-4d27-a428-6b286c339961 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 949 time: 0.0825937 May 31 11:58:09.445293 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-159d6e35-dd5d-4996-b0d1-40d5c4dd645b req-d0770d49-c86d-47a2-ae42-a375f3fd0d3d tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0568252 May 31 11:58:09.521349 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db0863f6-0eb1-4952-bbb2-854911b447c0 req-04d565a6-3fb5-4c55-a07f-20666ea5231f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:09.525515 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0863f6-0eb1-4952-bbb2-854911b447c0 req-04d565a6-3fb5-4c55-a07f-20666ea5231f service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 1102 time: 0.0780263 May 31 11:58:09.567595 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db0863f6-0eb1-4952-bbb2-854911b447c0 req-47de011d-a431-4fed-bb28-873474d2ff8f service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0370150 May 31 11:58:10.008967 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-89433d74-f5da-4eb1-9ea5-492ddc307189 req-4629ee47-f010-4bf6-9554-262f01f5d674 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:10.010729 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-89433d74-f5da-4eb1-9ea5-492ddc307189 req-4629ee47-f010-4bf6-9554-262f01f5d674 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0848582 May 31 11:58:10.040183 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b691af3-f868-4858-8076-4a0558c00671 req-7ad32131-4352-4794-95f0-1e44f6b9062a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:10.041660 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0b691af3-f868-4858-8076-4a0558c00671 req-7ad32131-4352-4794-95f0-1e44f6b9062a tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0831277 May 31 11:58:10.059875 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-89433d74-f5da-4eb1-9ea5-492ddc307189 req-3513ecf5-ede6-4b09-8916-918a83ed4a3b tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0428147 May 31 11:58:10.087882 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0b691af3-f868-4858-8076-4a0558c00671 req-b27355a6-c5fb-4ec3-b9b7-d3d384c407d3 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401893 May 31 11:58:10.403971 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4eb30d58-e5b6-499b-b9b7-ef393cfa6b1b req-5b235a40-8676-4dee-abda-c394aeb3faf8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:10.405413 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4eb30d58-e5b6-499b-b9b7-ef393cfa6b1b req-5b235a40-8676-4dee-abda-c394aeb3faf8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 946 time: 0.0919597 May 31 11:58:10.452837 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4eb30d58-e5b6-499b-b9b7-ef393cfa6b1b req-d931b2c7-7f4d-4ac6-b527-c931fc7472cd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0405793 May 31 11:58:10.463219 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 150f3a3b-28b5-435a-b826-0eee61f6b143 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:10.522684 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 150f3a3b-28b5-435a-b826-0eee61f6b143 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:58:10.522939 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 150f3a3b-28b5-435a-b826-0eee61f6b143 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:58:10.643380 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5b7cd01-79de-4ab7-b45f-7bf69bbac7e4 req-9af58ab8-865b-4546-95a3-3742727dcc91 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:10.645877 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c5b7cd01-79de-4ab7-b45f-7bf69bbac7e4 req-9af58ab8-865b-4546-95a3-3742727dcc91 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 949 time: 0.0867786 May 31 11:58:10.698488 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c5b7cd01-79de-4ab7-b45f-7bf69bbac7e4 req-32b82746-4e6e-4a93-b8aa-2dbe9dc49d97 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0460746 May 31 11:58:10.779918 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7fe2fc43-8791-453c-9feb-150696387348 req-5a6ceb1e-448c-4637-9238-9a4a3f12b5d3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:10.781334 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7fe2fc43-8791-453c-9feb-150696387348 req-5a6ceb1e-448c-4637-9238-9a4a3f12b5d3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 946 time: 0.0842626 May 31 11:58:10.826064 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7fe2fc43-8791-453c-9feb-150696387348 req-44524a22-bc66-4195-84f1-efc5c821d061 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0381496 May 31 11:58:11.276196 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e4897cd-c475-44e6-8a74-77cba54868c6 req-e01a406f-16b4-4ef9-9919-4218972671e8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:11.277880 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1e4897cd-c475-44e6-8a74-77cba54868c6 req-e01a406f-16b4-4ef9-9919-4218972671e8 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.0817206 May 31 11:58:11.315180 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67013217-28dd-4306-9f76-d95c0fa16ecf req-0d199aac-7c75-4acf-9fea-6568e1e993be tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:11.316800 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-67013217-28dd-4306-9f76-d95c0fa16ecf req-0d199aac-7c75-4acf-9fea-6568e1e993be tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.1397872 May 31 11:58:11.322836 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1e4897cd-c475-44e6-8a74-77cba54868c6 req-32ac50fa-6bb4-4608-b5e1-cbc3d571bde5 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/security-groups?id=9430886b-049d-489c-b4b2-fe5fa25dfff3&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0392663 May 31 11:58:11.324301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-5989d018-9d85-41e9-91c7-e2c466fd7d45 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:11.325704 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-5989d018-9d85-41e9-91c7-e2c466fd7d45 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.47 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 946 time: 0.1174567 May 31 11:58:11.422968 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-67013217-28dd-4306-9f76-d95c0fa16ecf req-8a3f5441-0c2b-4698-93fa-952c4600f2bb tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0962679 May 31 11:58:11.492356 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Deleting port f18f632c-d53b-4423-93a7-f93c3e8e0778 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:11.492704 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:11.628251 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] For port f18f632c-d53b-4423-93a7-f93c3e8e0778, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f18f632c-d53b-4423-93a7-f93c3e8e0778,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:11.628629 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:11.655718 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6778bdbc-9c1f-45b6-b5bc-9527553ec55b req-31ff4ab0-e42a-4f1e-aafd-078c18b4970f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:11.657162 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6778bdbc-9c1f-45b6-b5bc-9527553ec55b req-31ff4ab0-e42a-4f1e-aafd-078c18b4970f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 946 time: 0.0845919 May 31 11:58:11.701862 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6778bdbc-9c1f-45b6-b5bc-9527553ec55b req-4a3b3715-fa9b-4989-a367-fc1da0627350 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0380585 May 31 11:58:11.757923 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Calling delete_port for f18f632c-d53b-4423-93a7-f93c3e8e0778 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:11.938495 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-949a36a1-e7f4-4a17-a4a0-817f44794cb5 req-bb4306b0-3a21-4a75-bad0-8098ad777e43 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:11.940063 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-949a36a1-e7f4-4a17-a4a0-817f44794cb5 req-bb4306b0-3a21-4a75-bad0-8098ad777e43 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 949 time: 0.1059217 May 31 11:58:11.986868 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-949a36a1-e7f4-4a17-a4a0-817f44794cb5 req-f7cf1ef6-9326-492c-a82b-d235780ccfaa tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403583 May 31 11:58:11.988894 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:11.997590 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:12.006576 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['f18f632c-d53b-4423-93a7-f93c3e8e0778'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:58:12.016537 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e28bce29-b0dd-4194-8209-cd93177747dd req-98f02ae4-9c6b-4098-9605-272fd6929829 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:12.019769 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e28bce29-b0dd-4194-8209-cd93177747dd req-98f02ae4-9c6b-4098-9605-272fd6929829 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 946 time: 0.0846996 May 31 11:58:12.060866 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Pushing event deleted for resources: {'Port': ['ID=f18f632c-d53b-4423-93a7-f93c3e8e0778,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:12.063863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:12.079980 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e28bce29-b0dd-4194-8209-cd93177747dd req-1c302d1f-be8a-4a80-aeca-b3d9ead28366 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0493855 May 31 11:58:12.214768 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] No DVR routers for this DVR port f18f632c-d53b-4423-93a7-f93c3e8e0778 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:58:12.217253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:12.219828 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '66c4d526-5295-45db-9e0b-516b82570603', 'name': 'network-vif-deleted', 'tag': 'f18f632c-d53b-4423-93a7-f93c3e8e0778'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:12.222127 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-2754de15-7de1-4daf-afea-3410f30012a5" -d '{"events": [{"server_uuid": "66c4d526-5295-45db-9e0b-516b82570603", "name": "network-vif-deleted", "tag": "f18f632c-d53b-4423-93a7-f93c3e8e0778"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:12.268603 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:12 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-6e3cdfed-0bbf-4e4d-95cd-4517e5e544fb x-openstack-request-id: req-6e3cdfed-0bbf-4e4d-95cd-4517e5e544fb {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:12.272505 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "66c4d526-5295-45db-9e0b-516b82570603", "name": "network-vif-deleted", "tag": "f18f632c-d53b-4423-93a7-f93c3e8e0778", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:12.272505 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-6e3cdfed-0bbf-4e4d-95cd-4517e5e544fb {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:12.272505 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '66c4d526-5295-45db-9e0b-516b82570603', 'name': 'network-vif-deleted', 'tag': 'f18f632c-d53b-4423-93a7-f93c3e8e0778', 'status': 'completed', 'code': 200} May 31 11:58:12.342168 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-edffb1b1-9e85-474f-9d41-3ac813804cd7 req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.47 "DELETE /v2.0/ports/f18f632c-d53b-4423-93a7-f93c3e8e0778 HTTP/1.1" status: 204 len: 173 time: 0.9976246 May 31 11:58:12.482063 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c3051c22-26cc-4bd1-b1a7-60de6f6bea74 req-940f57b4-c573-4717-9df2-110f841b86b2 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] 10.222.0.48 "GET /v2.0/ports?device_id=66c4d526-5295-45db-9e0b-516b82570603 HTTP/1.1" status: 200 len: 210 time: 0.0472119 May 31 11:58:12.605735 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-c46f9eed-23a7-4714-8084-ac333f55ac04 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:12.606432 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-c46f9eed-23a7-4714-8084-ac333f55ac04 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:58:12.669158 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80b39396-437c-4e1a-8c12-2bf325a7c0de req-9e3aabd3-22e6-49f5-9c42-fe7c8a7d09bb tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:12.670715 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-80b39396-437c-4e1a-8c12-2bf325a7c0de req-9e3aabd3-22e6-49f5-9c42-fe7c8a7d09bb tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0819237 May 31 11:58:12.709095 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-c46f9eed-23a7-4714-8084-ac333f55ac04 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:58:12.710253 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-c46f9eed-23a7-4714-8084-ac333f55ac04 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 946 time: 0.1988726 May 31 11:58:12.721399 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-80b39396-437c-4e1a-8c12-2bf325a7c0de req-3db74d23-402b-430e-9452-0c761c311070 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0443015 May 31 11:58:12.783376 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6cf6e9d5-4014-4c87-88f3-c6463b928824 req-04788aa3-ec62-4bd6-a71e-bd4a34313b4f tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:12.784793 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6cf6e9d5-4014-4c87-88f3-c6463b928824 req-04788aa3-ec62-4bd6-a71e-bd4a34313b4f tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 949 time: 0.1373246 May 31 11:58:12.880243 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6cf6e9d5-4014-4c87-88f3-c6463b928824 req-b2ecd15b-eb76-4ac5-9464-0a3e9c0bd4a8 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0894239 May 31 11:58:12.897633 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Deleting port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:12.897976 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:13.029309 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] For port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:13.029685 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:13.155509 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Calling delete_port for 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:13.342871 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e96097f-3ed5-40fc-a46e-1d8c80422573 req-f3cbc4f2-0711-493d-9cf9-6d3c63aa43f7 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:13.344227 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5e96097f-3ed5-40fc-a46e-1d8c80422573 req-f3cbc4f2-0711-493d-9cf9-6d3c63aa43f7 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 946 time: 0.1517825 May 31 11:58:13.358563 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:13.365009 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:13.371835 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['c4a233f2-9446-42a7-964d-1dc2496c938a', '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:58:13.390447 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5e96097f-3ed5-40fc-a46e-1d8c80422573 req-be1ec66a-9b86-4784-8590-241612b437b0 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396318 May 31 11:58:13.410423 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Pushing event deleted for resources: {'Port': ['ID=064a6ea1-ceec-462e-b73a-4e2c88f7a5b8,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:13.413469 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:13.487683 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] No DVR routers for this DVR port 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:58:13.490601 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:13.579949 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34586) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:13.638456 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1646f9b8-5080-4b82-b614-3c66bd5aafee req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "DELETE /v2.0/ports/064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 HTTP/1.1" status: 204 len: 173 time: 0.9219677 May 31 11:58:13.929956 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c13b9390-e56f-4249-9336-59c7163f6bd4 req-eb2146f5-43f4-4db1-9ab0-74eed788e92f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:13.940079 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c13b9390-e56f-4249-9336-59c7163f6bd4 req-eb2146f5-43f4-4db1-9ab0-74eed788e92f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.1098976 May 31 11:58:13.986952 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c13b9390-e56f-4249-9336-59c7163f6bd4 req-13158dbc-1d38-497e-82e3-53d146c6192a tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0451639 May 31 11:58:14.276892 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '85c447cc-29b9-4fd4-a3f6-d7219276b581', 'name': 'network-vif-deleted', 'tag': '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:14.279272 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-2c76dfe8-8560-4f13-b889-a37d86043e86" -d '{"events": [{"server_uuid": "85c447cc-29b9-4fd4-a3f6-d7219276b581", "name": "network-vif-deleted", "tag": "064a6ea1-ceec-462e-b73a-4e2c88f7a5b8"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:14.311473 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b96b24c3-0143-481d-8e8d-0d050fc163e8 req-73bc9915-a50a-4a43-9742-7481577b4dbb tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:14.313032 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b96b24c3-0143-481d-8e8d-0d050fc163e8 req-73bc9915-a50a-4a43-9742-7481577b4dbb tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 949 time: 0.3092906 May 31 11:58:14.325345 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:14 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-f36d717c-fa40-4ba9-931b-1fdaefad407a x-openstack-request-id: req-f36d717c-fa40-4ba9-931b-1fdaefad407a {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:14.325678 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "85c447cc-29b9-4fd4-a3f6-d7219276b581", "name": "network-vif-deleted", "tag": "064a6ea1-ceec-462e-b73a-4e2c88f7a5b8", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:14.325951 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-f36d717c-fa40-4ba9-931b-1fdaefad407a {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:14.326348 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '85c447cc-29b9-4fd4-a3f6-d7219276b581', 'name': 'network-vif-deleted', 'tag': '064a6ea1-ceec-462e-b73a-4e2c88f7a5b8', 'status': 'completed', 'code': 200} May 31 11:58:14.368634 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-f45c668f-f6b3-4891-9c9d-6137c43270f6 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:14.370139 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-f45c668f-f6b3-4891-9c9d-6137c43270f6 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.47 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 949 time: 0.2794776 May 31 11:58:14.472663 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device f18f632c-d53b-4423-93a7-f93c3e8e0778 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:14.477328 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b96b24c3-0143-481d-8e8d-0d050fc163e8 req-3cd8f945-9d1e-4ebc-9e25-f2730bec5acc tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/security-groups?id=c99cbc4e-0997-40d0-9734-767b70686311&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1557250 May 31 11:58:14.555795 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: f18f632c-d53b-4423-93a7-f93c3e8e0778 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:58:14.556143 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device f18f632c-d53b-4423-93a7-f93c3e8e0778 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:58:14.556562 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:14.588392 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-fc8a3575-9432-46e3-8453-5b5ee2a38847 req-913e85be-2b7f-48c4-8252-51d8427e4833 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=85c447cc-29b9-4fd4-a3f6-d7219276b581 HTTP/1.1" status: 200 len: 210 time: 0.0506537 May 31 11:58:14.594355 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:58:14.598635 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 064a6ea1-ceec-462e-b73a-4e2c88f7a5b8 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:58:14.598635 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device cc7a1afb-a9a6-4702-81b7-99edecc48d2c no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:14.601719 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:14.673027 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Deleting port cc7a1afb-a9a6-4702-81b7-99edecc48d2c {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:14.673364 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:14.725262 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:14.755004 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:14.757333 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-5599d093-6574-4c23-ae42-ec6f4b2a912f" -d '{"events": [{"server_uuid": "d7675d4e-5c62-4304-b0fc-603206d7a8e6", "name": "network-vif-unplugged", "status": "completed", "tag": "cc7a1afb-a9a6-4702-81b7-99edecc48d2c"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:14.765697 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port cc7a1afb-a9a6-4702-81b7-99edecc48d2c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,segment=NetworkSegment(a7256a82-57c5-4819-8aea-511768fbafdc),segment_id=a7256a82-57c5-4819-8aea-511768fbafdc)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:14.813194 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 185 Content-Type: application/json Date: Mon, 31 May 2021 11:58:14 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-b0c0b162-cb32-4649-918e-75b6d73c275c x-openstack-request-id: req-b0c0b162-cb32-4649-918e-75b6d73c275c {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:14.813194 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "d7675d4e-5c62-4304-b0fc-603206d7a8e6", "name": "network-vif-unplugged", "status": "completed", "tag": "cc7a1afb-a9a6-4702-81b7-99edecc48d2c", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:14.813194 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-b0c0b162-cb32-4649-918e-75b6d73c275c {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:14.813194 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c', 'code': 200} May 31 11:58:14.864283 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:14.867106 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:14.937335 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,revision_number=6']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:14.940570 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:14.993408 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] For port cc7a1afb-a9a6-4702-81b7-99edecc48d2c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,segment=NetworkSegment(a7256a82-57c5-4819-8aea-511768fbafdc),segment_id=a7256a82-57c5-4819-8aea-511768fbafdc)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:14.993793 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:15.124782 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Calling delete_port for cc7a1afb-a9a6-4702-81b7-99edecc48d2c owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:15.177401 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-33fb634a-9e86-4afd-9a50-565f8f8d28a8 req-115456e5-4365-4538-bb50-67e7e5e362e6 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:15.178873 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-33fb634a-9e86-4afd-9a50-565f8f8d28a8 req-115456e5-4365-4538-bb50-67e7e5e362e6 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0805814 May 31 11:58:15.230000 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-33fb634a-9e86-4afd-9a50-565f8f8d28a8 req-4fd10bfd-19b2-49bd-8ec7-8635b54cda54 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0444012 May 31 11:58:15.349235 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Deleting port b808043f-df25-4425-a3f4-6ee4d3a2150f {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:15.349586 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:15.413550 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:15.422464 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0e55596b-ac0f-4d63-80c1-090c08d6eaff tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:15.473329 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Pushing event deleted for resources: {'Port': ['ID=cc7a1afb-a9a6-4702-81b7-99edecc48d2c,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:15.478377 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:15.568823 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] No DVR routers for this DVR port cc7a1afb-a9a6-4702-81b7-99edecc48d2c on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:58:15.573136 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:15.599246 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] For port b808043f-df25-4425-a3f4-6ee4d3a2150f, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b808043f-df25-4425-a3f4-6ee4d3a2150f,segment=NetworkSegment(8a1730f3-5150-4597-9255-89922dd429cc),segment_id=8a1730f3-5150-4597-9255-89922dd429cc)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:15.599589 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:15.703675 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c81f7801-baf0-4163-a5a3-39ba99398840 req-6f58ff9b-1d5a-4e28-a316-d7ec6d5a0c0c tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Synchronizing usage tracker for tenant:d4b0a729336241e29d7d74dcbc6dc060 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:58:15.851817 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d258cac4-6cac-4033-ac7c-2aa47d278a9f req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.47 "DELETE /v2.0/ports/cc7a1afb-a9a6-4702-81b7-99edecc48d2c HTTP/1.1" status: 204 len: 173 time: 1.4723372 May 31 11:58:15.858467 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Calling delete_port for b808043f-df25-4425-a3f4-6ee4d3a2150f owned by network:router_interface {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:16.081492 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c81f7801-baf0-4163-a5a3-39ba99398840 req-6f58ff9b-1d5a-4e28-a316-d7ec6d5a0c0c tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Unset dirty status for tenant:d4b0a729336241e29d7d74dcbc6dc060 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:58:16.082577 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c81f7801-baf0-4163-a5a3-39ba99398840 req-6f58ff9b-1d5a-4e28-a316-d7ec6d5a0c0c tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] 10.222.0.48 "GET /v2.0/ports?device_id=d7675d4e-5c62-4304-b0fc-603206d7a8e6 HTTP/1.1" status: 200 len: 210 time: 0.4756320 May 31 11:58:16.264477 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:16.270154 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:16.313667 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Pushing event deleted for resources: {'Port': ['ID=b808043f-df25-4425-a3f4-6ee4d3a2150f,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:16.318429 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:16.349391 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'name': 'network-vif-deleted', 'tag': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:16.351855 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-3fc51860-bc1a-4f68-bc2d-b91818f0bcc8" -d '{"events": [{"server_uuid": "d7675d4e-5c62-4304-b0fc-603206d7a8e6", "name": "network-vif-deleted", "tag": "cc7a1afb-a9a6-4702-81b7-99edecc48d2c"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:16.416417 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:16 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-eab31931-7fda-42a9-925c-d2ca60ff7f7a x-openstack-request-id: req-eab31931-7fda-42a9-925c-d2ca60ff7f7a {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:16.417119 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "d7675d4e-5c62-4304-b0fc-603206d7a8e6", "name": "network-vif-deleted", "tag": "cc7a1afb-a9a6-4702-81b7-99edecc48d2c", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:16.417119 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-eab31931-7fda-42a9-925c-d2ca60ff7f7a {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:16.417474 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd7675d4e-5c62-4304-b0fc-603206d7a8e6', 'name': 'network-vif-deleted', 'tag': 'cc7a1afb-a9a6-4702-81b7-99edecc48d2c', 'status': 'completed', 'code': 200} May 31 11:58:16.432349 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4d260ad-849c-4d47-8318-d66bc5edeccb req-da9389a5-5f11-456d-b56f-c2940490217d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:16.433837 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f4d260ad-849c-4d47-8318-d66bc5edeccb req-da9389a5-5f11-456d-b56f-c2940490217d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0888450 May 31 11:58:16.480706 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-f4d260ad-849c-4d47-8318-d66bc5edeccb req-fb8d76af-b164-44c8-97a1-921079900202 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0409327 May 31 11:58:16.555295 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:16.916605 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Router 402aeb18-0749-46be-8b62-c6de2c58aad6 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:58:17.095428 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:58:17.104828 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] Sync routers for ids ['402aeb18-0749-46be-8b62-c6de2c58aad6'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:58:17.108733 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] 10.222.0.48 "PUT /v2.0/routers/402aeb18-0749-46be-8b62-c6de2c58aad6/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.5275326 May 31 11:58:17.111199 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34618) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:17.118622 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] Routers {'402aeb18-0749-46be-8b62-c6de2c58aad6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:58:17.119315 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:57:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['402aeb18-0749-46be-8b62-c6de2c58aad6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:17.300821 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks [] for router, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:17.301282 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] Set router 402aeb18-0749-46be-8b62-c6de2c58aad6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:58:17.490543 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:17.612932 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:58:16Z', 'revision_number': 5, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:36Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:17.626879 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] Checking router: 402aeb18-0749-46be-8b62-c6de2c58aad6 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:58:17.626879 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] Routers {'402aeb18-0749-46be-8b62-c6de2c58aad6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:58:17.627479 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:57:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['402aeb18-0749-46be-8b62-c6de2c58aad6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:17.688158 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3dc75ef6-a20f-4627-9bc9-16208f744f1f req-bef59dd7-8260-46c4-97b8-a688111b6d26 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:17.689817 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3dc75ef6-a20f-4627-9bc9-16208f744f1f req-bef59dd7-8260-46c4-97b8-a688111b6d26 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0971487 May 31 11:58:17.742713 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3dc75ef6-a20f-4627-9bc9-16208f744f1f req-340b3fd9-07a8-4451-8498-08052a2a53e5 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0440867 May 31 11:58:17.768269 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Deleting port 3778e8e5-5392-49be-a3b1-1b01bb388e8d {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:17.768861 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:17.818173 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] Set router 402aeb18-0749-46be-8b62-c6de2c58aad6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:58:17.906777 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] For port 3778e8e5-5392-49be-a3b1-1b01bb388e8d, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=3778e8e5-5392-49be-a3b1-1b01bb388e8d,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:17.907265 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:18.042020 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Calling delete_port for 3778e8e5-5392-49be-a3b1-1b01bb388e8d owned by network:router_gateway {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:18.135496 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc011eb5-577d-43e6-b470-04d8c9c27e60 req-84a34784-2054-4d03-a761-508f7456fa2d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:18.136516 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-dc011eb5-577d-43e6-b470-04d8c9c27e60 req-84a34784-2054-4d03-a761-508f7456fa2d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Synchronizing usage tracker for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:58:18.149343 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:58:16Z', 'revision_number': 5, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:36Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:18.153425 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a11deb38-2bb7-4038-9b18-e6ef1d82f490 None None] The sync data for ids ['402aeb18-0749-46be-8b62-c6de2c58aad6'] in dv-cinder-791239-4-iscsi is: [{'id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'name': 'tempest-ServerRescueNegativeTestJSON-1966173874-router', 'tenant_id': 'c94978b38250481095d2824adfb16ed4', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64'}], 'enable_snat': True}, 'gw_port_id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:58:16Z', 'revision_number': 5, 'project_id': 'c94978b38250481095d2824adfb16ed4', 'gw_port': {'id': '3778e8e5-5392-49be-a3b1-1b01bb388e8d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:6b:8d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '402aeb18-0749-46be-8b62-c6de2c58aad6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.64', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:20Z', 'updated_at': '2021-05-31T11:54:36Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:58:18.297277 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-dc011eb5-577d-43e6-b470-04d8c9c27e60 req-84a34784-2054-4d03-a761-508f7456fa2d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Unset dirty status for tenant:7dea63b6c4a340d7b361fe91ca6b8281 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:58:18.302247 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc011eb5-577d-43e6-b470-04d8c9c27e60 req-84a34784-2054-4d03-a761-508f7456fa2d tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 945 time: 0.2537622 May 31 11:58:18.342585 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc011eb5-577d-43e6-b470-04d8c9c27e60 req-c4d9880c-3a9c-4237-9776-f7da0fffb2f2 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394917 May 31 11:58:18.389775 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:18.395334 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:18.521428 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Pushing event deleted for resources: {'Port': ['ID=3778e8e5-5392-49be-a3b1-1b01bb388e8d,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:18.548376 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.153s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:18.629267 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:18.636318 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b808043f-df25-4425-a3f4-6ee4d3a2150f no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:18.672666 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: b808043f-df25-4425-a3f4-6ee4d3a2150f {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:58:18.673013 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b808043f-df25-4425-a3f4-6ee4d3a2150f not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:58:18.822326 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:18.826706 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:18.989556 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93c0b73a-5211-4393-89d2-f02f51331abd req-e74df8a0-9c27-4f1c-9b49-c20e61b51124 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:18.991089 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-93c0b73a-5211-4393-89d2-f02f51331abd req-e74df8a0-9c27-4f1c-9b49-c20e61b51124 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0915806 May 31 11:58:19.037835 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-93c0b73a-5211-4393-89d2-f02f51331abd req-2e7ffac1-d9ed-48e4-93a7-b67146617cfe tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0409386 May 31 11:58:19.104013 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.104549 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 402aeb18-0749-46be-8b62-c6de2c58aad6 {{(pid=99039) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 11:58:19.118356 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.119549 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-5978c8ec-5b14-4944-a09c-f6c5a37027c7 admin admin] 10.222.0.48 "DELETE /v2.0/routers/402aeb18-0749-46be-8b62-c6de2c58aad6 HTTP/1.1" status: 204 len: 168 time: 2.0070534 May 31 11:58:19.122083 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34716) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:19.186237 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Deleting subnet 6312c1ab-b503-4bcc-af02-8274dbf2679c {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 11:58:19.191157 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.248841 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.307237 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.667261 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.672541 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-dffa71aa-101f-4274-a7eb-f410543f834d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:19.676375 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d217bd6f-b887-4381-98e2-2e11bed23842 req-30512766-33da-403f-8ce4-474ad7d34c0b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:19.677967 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d217bd6f-b887-4381-98e2-2e11bed23842 req-30512766-33da-403f-8ce4-474ad7d34c0b tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 945 time: 0.2187450 May 31 11:58:19.708700 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=6312c1ab-b503-4bcc-af02-8274dbf2679c,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:19.710686 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.714541 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:19.717591 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/6312c1ab-b503-4bcc-af02-8274dbf2679c HTTP/1.1" status: 204 len: 168 time: 0.5943758 May 31 11:58:19.719880 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34738) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:19.766557 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d217bd6f-b887-4381-98e2-2e11bed23842 req-11db0ac6-15fb-44ff-b7d7-78230d015c26 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/security-groups?id=9227c84c-4b80-455f-bb10-219156b82a20&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0822270 May 31 11:58:19.807032 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-169cfccf-cc7e-42a3-bffe-5c00cfc4143f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:19.808994 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-169cfccf-cc7e-42a3-bffe-5c00cfc4143f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 945 time: 0.0916240 May 31 11:58:19.825920 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.953247 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Deleting port c4a233f2-9446-42a7-964d-1dc2496c938a {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:19.953610 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.954421 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:19.967329 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['c36bb4a9-2822-49b1-b291-69604a881999']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:19.983707 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '8a1730f3-5150-4597-9255-89922dd429cc') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:19.984000 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '8a1730f3-5150-4597-9255-89922dd429cc') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:19.995854 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.013095 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.020350 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Releasing vlan 387 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 11:58:20.020889 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 11:58:20.043903 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 11:58:20.108465 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] For port c4a233f2-9446-42a7-964d-1dc2496c938a, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c4a233f2-9446-42a7-964d-1dc2496c938a,segment=NetworkSegment(a846ec5a-b6ef-4a36-83ff-119eb454231c),segment_id=a846ec5a-b6ef-4a36-83ff-119eb454231c)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:20.108727 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.218619 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.234819 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.236559 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:20.260410 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Calling delete_port for c4a233f2-9446-42a7-964d-1dc2496c938a owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:20.267133 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Pushing event deleted for resources: {'Network': ['ID=c36bb4a9-2822-49b1-b291-69604a881999,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:20.281810 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:20.347102 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] 10.222.0.48 "DELETE /v2.0/networks/c36bb4a9-2822-49b1-b291-69604a881999 HTTP/1.1" status: 204 len: 168 time: 0.6239965 May 31 11:58:20.469935 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.483255 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['c4a233f2-9446-42a7-964d-1dc2496c938a'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:58:20.484022 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e095fb47-24d6-41fb-bc86-3f11c626f379 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:20.532870 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b116666a-7deb-4a96-a2cb-583d1f0a8715 req-92a8abf1-6242-47bc-b0e7-a61869543444 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:20.534007 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b116666a-7deb-4a96-a2cb-583d1f0a8715 req-92a8abf1-6242-47bc-b0e7-a61869543444 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.1739550 May 31 11:58:20.547474 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 3778e8e5-5392-49be-a3b1-1b01bb388e8d no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:20.586540 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Pushing event deleted for resources: {'Port': ['ID=c4a233f2-9446-42a7-964d-1dc2496c938a,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:20.591084 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 3778e8e5-5392-49be-a3b1-1b01bb388e8d {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:58:20.591574 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b116666a-7deb-4a96-a2cb-583d1f0a8715 req-e3046abb-d463-4854-ac2b-7fa1576690f0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0494990 May 31 11:58:20.593661 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 3778e8e5-5392-49be-a3b1-1b01bb388e8d not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:58:20.630271 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.630652 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.631037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.631598 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.631973 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.632333 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.632698 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.633058 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.633415 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.633850 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.634204 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.634559 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:20.638131 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] No DVR routers for this DVR port c4a233f2-9446-42a7-964d-1dc2496c938a on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:58:20.639654 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:20.640733 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.644615 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bb55be8b-198d-40e7-849a-8543fbf35000', 'name': 'network-vif-deleted', 'tag': 'c4a233f2-9446-42a7-964d-1dc2496c938a'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:20.646423 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-15313311-36f7-4f9f-87be-483bd23ccc8d" -d '{"events": [{"server_uuid": "bb55be8b-198d-40e7-849a-8543fbf35000", "name": "network-vif-deleted", "tag": "c4a233f2-9446-42a7-964d-1dc2496c938a"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:20.689402 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:20 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-a09332c3-98df-41ec-ad01-5325181cfd65 x-openstack-request-id: req-a09332c3-98df-41ec-ad01-5325181cfd65 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:20.690074 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "bb55be8b-198d-40e7-849a-8543fbf35000", "name": "network-vif-deleted", "tag": "c4a233f2-9446-42a7-964d-1dc2496c938a", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:20.690074 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-a09332c3-98df-41ec-ad01-5325181cfd65 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:20.690556 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bb55be8b-198d-40e7-849a-8543fbf35000', 'name': 'network-vif-deleted', 'tag': 'c4a233f2-9446-42a7-964d-1dc2496c938a', 'status': 'completed', 'code': 200} May 31 11:58:20.848551 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e2f4cc42-de3f-4d8b-96b8-8ea611314a9c req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.47 "DELETE /v2.0/ports/c4a233f2-9446-42a7-964d-1dc2496c938a HTTP/1.1" status: 204 len: 173 time: 1.0298419 May 31 11:58:20.899190 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-aa060b66-747c-4851-9505-d360bfb2fe30 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:20.900643 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34808) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:20.921281 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bb605add-5c97-4fa8-9c19-8bce67a546fb req-c58ce7c3-6210-4c5f-b921-810c3e67eb1a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] 10.222.0.48 "GET /v2.0/ports?device_id=bb55be8b-198d-40e7-849a-8543fbf35000 HTTP/1.1" status: 200 len: 210 time: 0.0480824 May 31 11:58:20.956551 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-34c0e013-271d-4251-9056-8f6baf6c27d9 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=c94978b38250481095d2824adfb16ed4&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0536954 May 31 11:58:20.959156 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34812) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:21.022308 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4c51b31a-6ccf-4858-8536-e7153000532c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:21.050083 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4c51b31a-6ccf-4858-8536-e7153000532c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:21.218630 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4c51b31a-6ccf-4858-8536-e7153000532c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:21.224162 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3f3a538c-8856-41b7-9541-5bcfd965c073 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:21.236949 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4c51b31a-6ccf-4858-8536-e7153000532c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9430886b-049d-489c-b4b2-fe5fa25dfff3,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:21.242333 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4c51b31a-6ccf-4858-8536-e7153000532c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:21.357431 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4c51b31a-6ccf-4858-8536-e7153000532c admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:21.358624 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-4c51b31a-6ccf-4858-8536-e7153000532c admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/9430886b-049d-489c-b4b2-fe5fa25dfff3 HTTP/1.1" status: 204 len: 168 time: 0.3985815 May 31 11:58:21.473217 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:58:21.798556 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d62780d-eb70-47fc-9e17-8d47c3510cc1 req-440d07df-03fe-47f2-ba0d-3ec8903c718d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:21.806281 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1d62780d-eb70-47fc-9e17-8d47c3510cc1 req-440d07df-03fe-47f2-ba0d-3ec8903c718d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0985131 May 31 11:58:21.861702 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1d62780d-eb70-47fc-9e17-8d47c3510cc1 req-1e6c94c7-b971-46f0-aa98-fbfbbbcdbeae tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0545218 May 31 11:58:22.025447 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34870) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:22.092361 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_v2 [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] Deleting subnet 4c58bc8a-d411-461e-a483-39773ff0d6f4 {{(pid=99040) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 11:58:22.092361 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:22.142057 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:22.154512 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:22.502297 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device c4a233f2-9446-42a7-964d-1dc2496c938a no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:58:22.537399 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012'] for subnet, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:22.543762 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:22.544213 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: c4a233f2-9446-42a7-964d-1dc2496c938a {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:58:22.545113 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device c4a233f2-9446-42a7-964d-1dc2496c938a not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:58:22.579162 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] Pushing event deleted for resources: {'Subnet': ['ID=4c58bc8a-d411-461e-a483-39773ff0d6f4,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:22.582338 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:22.588716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:22.594816 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-cb219bf6-0f98-4661-b388-e3b91ea961cb admin admin] 10.222.0.48 "DELETE /v2.0/subnets/4c58bc8a-d411-461e-a483-39773ff0d6f4 HTTP/1.1" status: 204 len: 168 time: 0.5683401 May 31 11:58:22.597903 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34912) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:22.724971 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:22.972785 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:22.988683 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > segment before_delete 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > port after_create 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > port after_delete 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > port after_update 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > segment after_delete 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:22.992926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > subnet after_update 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:23.106311 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:23.106655 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:23.106978 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > subnet before_create 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:23.107302 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Subscribe: > subnet before_update 55550000 {{(pid=99040) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 31 11:58:23.107658 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['709626b6-807b-4256-9031-1578948f1fe8']}} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:23.154871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'a846ec5a-b6ef-4a36-83ff-119eb454231c') {'for_net_delete': True} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:23.155157 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'a846ec5a-b6ef-4a36-83ff-119eb454231c') {} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:23.177882 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3096822'] for segment, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:23.205125 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:23.215930 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Releasing vlan 356 on physical network physnet1 to pool {{(pid=99040) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 11:58:23.216308 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2278144'] for segment, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: from cryptography.utils import int_from_bytes May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: from cryptography.utils import int_from_bytes May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: Traceback (most recent call last): May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: timer() May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: cb(*args, **kw) May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: return func(*args, **kwargs) May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: self._notify() May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: self.callback(batched_events) May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: event.method(event) May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_uuid = self.p_client.list_aggregates( May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: return f(self, *a, **k) May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._get(url).json() May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: return self.request(url, 'GET', **kwargs) May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: base_url = self.get_endpoint(auth, allow=allow, May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 11:58:23.232569 dv-cinder-791239-4-iscsi neutron-server[99040]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 11:58:23.318676 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03016c15-3ae5-47cb-9833-ada8da2c4103 req-1e6c6de1-7467-4a0d-bd35-f65430031495 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:23.320220 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-03016c15-3ae5-47cb-9833-ada8da2c4103 req-1e6c6de1-7467-4a0d-bd35-f65430031495 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.2048225 May 31 11:58:23.368781 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-03016c15-3ae5-47cb-9833-ada8da2c4103 req-cc5b9520-39ea-4d05-9588-06ba5d28ac64 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0425286 May 31 11:58:23.399185 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:23.446828 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:23.451205 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:23.472911 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:23.475245 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e2c504c9-b807-4a1e-ba56-1b741fdd5467 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:23.475507 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Pushing event deleted for resources: {'Network': ['ID=709626b6-807b-4256-9031-1578948f1fe8,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:23.478382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:23.564580 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] 10.222.0.48 "DELETE /v2.0/networks/709626b6-807b-4256-9031-1578948f1fe8 HTTP/1.1" status: 204 len: 168 time: 0.9629250 May 31 11:58:24.113402 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34946) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:24.167233 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-ab443972-888d-4729-a2cd-7fc8024e007c admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=7dea63b6c4a340d7b361fe91ca6b8281&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0516951 May 31 11:58:24.169898 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 34952) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:24.232153 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1728ed31-06bc-494e-b35a-138e1e3ba750 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:24.259348 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1728ed31-06bc-494e-b35a-138e1e3ba750 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:24.481010 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1728ed31-06bc-494e-b35a-138e1e3ba750 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:24.485297 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c3ed7e56-52b6-4ab0-a98a-5910cc2232d9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:24.499334 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1728ed31-06bc-494e-b35a-138e1e3ba750 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9227c84c-4b80-455f-bb10-219156b82a20,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:24.502605 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1728ed31-06bc-494e-b35a-138e1e3ba750 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:24.581380 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96cdf3a8-5122-4d01-901e-a820c9518d61 req-7735b840-d09f-4d1a-9419-755698a482c4 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:24.584451 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-96cdf3a8-5122-4d01-901e-a820c9518d61 req-7735b840-d09f-4d1a-9419-755698a482c4 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.1078997 May 31 11:58:24.612923 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1728ed31-06bc-494e-b35a-138e1e3ba750 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:24.615022 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-1728ed31-06bc-494e-b35a-138e1e3ba750 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/9227c84c-4b80-455f-bb10-219156b82a20 HTTP/1.1" status: 204 len: 168 time: 0.4435186 May 31 11:58:24.637985 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-96cdf3a8-5122-4d01-901e-a820c9518d61 req-9385325d-e1cf-415d-943a-1f593bb2457c tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0490203 May 31 11:58:24.642118 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:24.740419 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ca109ae-0a79-4718-b57e-696cbd282e5b req-1bfbd1fa-6ab8-4c32-ac94-0bf808a6edcd tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:24.764469 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5ca109ae-0a79-4718-b57e-696cbd282e5b req-1bfbd1fa-6ab8-4c32-ac94-0bf808a6edcd tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1431022 May 31 11:58:24.810390 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34966) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:24.817649 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5ca109ae-0a79-4718-b57e-696cbd282e5b req-860512cc-7852-4718-83b2-2394d60f5067 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0521760 May 31 11:58:24.967767 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8d3938f7-767f-4182-b295-5163ff7bc95e admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=d4b0a729336241e29d7d74dcbc6dc060&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1559818 May 31 11:58:24.970297 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 34972) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:25.033692 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8dadc17a-8e97-4655-a690-04002b082587 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:25.059563 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8dadc17a-8e97-4655-a690-04002b082587 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:25.364264 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8dadc17a-8e97-4655-a690-04002b082587 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:25.370644 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-991e5232-f117-482b-ac23-429226170658 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:25.395667 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8dadc17a-8e97-4655-a690-04002b082587 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c99cbc4e-0997-40d0-9734-767b70686311,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:25.398763 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8dadc17a-8e97-4655-a690-04002b082587 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:25.485931 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8dadc17a-8e97-4655-a690-04002b082587 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:25.490030 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8dadc17a-8e97-4655-a690-04002b082587 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/c99cbc4e-0997-40d0-9734-767b70686311 HTTP/1.1" status: 204 len: 168 time: 0.5165546 May 31 11:58:25.852399 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2531b821-8799-4d7f-bde5-1d97e6aed866 req-729a63fd-59b8-47db-956d-3958035f2c67 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:25.853853 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2531b821-8799-4d7f-bde5-1d97e6aed866 req-729a63fd-59b8-47db-956d-3958035f2c67 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0929012 May 31 11:58:25.949754 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2531b821-8799-4d7f-bde5-1d97e6aed866 req-35b90668-d6fb-4069-8d5a-5b35dbc3ca0c tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=82252074-5d18-415f-aaff-3f993aa67988&id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0888646 May 31 11:58:26.091739 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e25e6070-18e6-4eb4-a6db-941014e7f1ea req-0951e9a2-9957-49b1-81e2-ac8d03af1938 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:26.093263 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e25e6070-18e6-4eb4-a6db-941014e7f1ea req-0951e9a2-9957-49b1-81e2-ac8d03af1938 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0857987 May 31 11:58:26.139570 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e25e6070-18e6-4eb4-a6db-941014e7f1ea req-cbf7c931-063b-4066-8f3f-b4526107bf30 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0401890 May 31 11:58:26.178394 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:27.337513 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2e7e565-5812-4051-b905-fb086e3b8467 req-3140dd4e-bada-4be1-9f0e-f5b3e362e720 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:27.339665 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d2e7e565-5812-4051-b905-fb086e3b8467 req-3140dd4e-bada-4be1-9f0e-f5b3e362e720 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0879285 May 31 11:58:27.385130 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d2e7e565-5812-4051-b905-fb086e3b8467 req-59aff395-7118-44f4-86a7-c7131020679d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0401599 May 31 11:58:28.570529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05592eab-a4c0-4606-a9d9-81eea3512258 req-a1e911a6-dd83-4002-aad5-dd083470aa01 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:28.571977 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-05592eab-a4c0-4606-a9d9-81eea3512258 req-a1e911a6-dd83-4002-aad5-dd083470aa01 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0813460 May 31 11:58:28.617306 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-05592eab-a4c0-4606-a9d9-81eea3512258 req-a2a9c21a-4c67-4796-b459-742b44c61bb8 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0396235 May 31 11:58:29.807718 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17166096-91f2-4e0c-aaac-0e0274b782c8 req-f5b99d35-1df7-4f6e-aade-4bf8c1ccf371 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:29.809153 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-17166096-91f2-4e0c-aaac-0e0274b782c8 req-f5b99d35-1df7-4f6e-aade-4bf8c1ccf371 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0837774 May 31 11:58:29.853255 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-17166096-91f2-4e0c-aaac-0e0274b782c8 req-3e642edb-c07e-407e-8170-e9e8fefdbcb3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0382340 May 31 11:58:30.334637 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35090) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:30.451105 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:30.466053 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:30.469924 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Usage tracker for resource:security_group_rule and tenant:3303eb9dfc95404b993a9dd7d8ed8274 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:30.473253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:30.479027 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:30.563253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:30.626676 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:30.635426 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:30.679842 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=3303eb9dfc95404b993a9dd7d8ed8274&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3441517 May 31 11:58:30.680723 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9f85c0de-8728-4e9e-bcce-cc06c6c8f041,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:30.682643 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35104) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:30.683670 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:30.739965 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:30.764602 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:30.929921 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:30.934815 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d25f0bf4-2855-43f2-801a-5d16bf9cc72e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:30.945444 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9f85c0de-8728-4e9e-bcce-cc06c6c8f041,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:30.956267 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.956630 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.957006 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.957324 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.957702 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.958033 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.962687 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.962687 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.962687 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.962687 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.962687 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.962687 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:58:30.963301 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:30.966462 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/9f85c0de-8728-4e9e-bcce-cc06c6c8f041 HTTP/1.1" status: 204 len: 168 time: 0.2804439 May 31 11:58:30.966462 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:31.041409 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-44231f80-28e5-4703-90fb-bb55a50cfd01 req-59963ec0-2148-4dfb-8c5f-7aeaf94d1a0f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:31.042874 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-44231f80-28e5-4703-90fb-bb55a50cfd01 req-59963ec0-2148-4dfb-8c5f-7aeaf94d1a0f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0842559 May 31 11:58:31.087872 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-44231f80-28e5-4703-90fb-bb55a50cfd01 req-f92d4e20-d513-4889-8bd3-9d99d94320d3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393395 May 31 11:58:31.645924 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35120) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:31.663590 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:31.678033 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:31.681790 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Usage tracker for resource:security_group_rule and tenant:3b2766585b74437eb0d2715093e6e1e4 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:31.684558 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:31.690766 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:31.773811 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:31.846863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:31.854148 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-03180dbb-5e01-438c-b295-739aeba7c9a2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:31.892787 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2a1349ae-af53-41e6-9c6b-5ae7c7c65a90,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:31.896522 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:31.898991 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=3b2766585b74437eb0d2715093e6e1e4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2520745 May 31 11:58:31.909058 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35128) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:31.967111 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4bf91d82-6f4b-47cb-aacf-607fc6794041 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:31.993818 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4bf91d82-6f4b-47cb-aacf-607fc6794041 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:32.141222 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4bf91d82-6f4b-47cb-aacf-607fc6794041 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:32.146885 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-57f19dca-da12-4ce6-bea6-c876fe67a29f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:32.160436 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4bf91d82-6f4b-47cb-aacf-607fc6794041 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2a1349ae-af53-41e6-9c6b-5ae7c7c65a90,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:32.164107 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4bf91d82-6f4b-47cb-aacf-607fc6794041 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:32.169813 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-4bf91d82-6f4b-47cb-aacf-607fc6794041 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:32.173168 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-4bf91d82-6f4b-47cb-aacf-607fc6794041 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/2a1349ae-af53-41e6-9c6b-5ae7c7c65a90 HTTP/1.1" status: 204 len: 168 time: 0.2608712 May 31 11:58:32.286608 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e5ce1dd-8f9b-4d33-8677-9627a5d9f950 req-a05695ab-8d5b-4771-8784-977477d54ac7 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:32.288109 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1e5ce1dd-8f9b-4d33-8677-9627a5d9f950 req-a05695ab-8d5b-4771-8784-977477d54ac7 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0873520 May 31 11:58:32.332669 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1e5ce1dd-8f9b-4d33-8677-9627a5d9f950 req-bc52885e-7a6b-4515-99d3-c7554e26a0c9 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0385425 May 31 11:58:33.524026 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-61db7f64-6c5d-44f6-9dfa-9493ca8ebfc0 req-04f5ac40-fe4f-4487-9c0b-ccda7b29a779 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:33.525451 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-61db7f64-6c5d-44f6-9dfa-9493ca8ebfc0 req-04f5ac40-fe4f-4487-9c0b-ccda7b29a779 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0840547 May 31 11:58:33.575196 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-61db7f64-6c5d-44f6-9dfa-9493ca8ebfc0 req-9a37b980-8ebc-4fc7-8ff7-982b04daaddb tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0435555 May 31 11:58:33.739461 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35184) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:33.857311 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:33.875271 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:33.879213 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Usage tracker for resource:security_group_rule and tenant:78bd2f8e6ab54558baa9a545198b68cd is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:33.882039 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:33.887735 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:33.969754 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:34.070216 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:34.078239 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a81e7546-5513-4114-b1f0-7d6e5d561705 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:34.129498 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=69a2cc76-61d8-4574-9622-9ea6fe2602b4,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:34.133843 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=78bd2f8e6ab54558baa9a545198b68cd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3933280 May 31 11:58:34.134836 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:34.138297 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35196) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:34.197964 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e156dd6-62f4-488e-98aa-5275282a6cd0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:34.230016 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e156dd6-62f4-488e-98aa-5275282a6cd0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:34.417938 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e156dd6-62f4-488e-98aa-5275282a6cd0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:34.421997 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-320e2fff-f12b-45bb-9950-30d8566048dd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:34.438934 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1e156dd6-62f4-488e-98aa-5275282a6cd0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=69a2cc76-61d8-4574-9622-9ea6fe2602b4,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:34.441403 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1e156dd6-62f4-488e-98aa-5275282a6cd0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:34.447637 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e156dd6-62f4-488e-98aa-5275282a6cd0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:34.451207 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1e156dd6-62f4-488e-98aa-5275282a6cd0 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/69a2cc76-61d8-4574-9622-9ea6fe2602b4 HTTP/1.1" status: 204 len: 168 time: 0.3112366 May 31 11:58:34.814014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03ad5690-1c54-4b85-b2bc-ab9e2063b4b2 req-599443fd-fad8-4ea7-8fb7-946b7c35ecda tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:34.816247 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-03ad5690-1c54-4b85-b2bc-ab9e2063b4b2 req-599443fd-fad8-4ea7-8fb7-946b7c35ecda tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1028664 May 31 11:58:34.879829 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-03ad5690-1c54-4b85-b2bc-ab9e2063b4b2 req-897e111b-47fd-4f72-805f-299378e1fb8a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0564430 May 31 11:58:36.238830 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1fb319c0-a1b5-496e-ad70-79a88e971a53 req-31da7b86-1a25-41d7-8eb7-d85e8700d17f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:36.240298 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1fb319c0-a1b5-496e-ad70-79a88e971a53 req-31da7b86-1a25-41d7-8eb7-d85e8700d17f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0940406 May 31 11:58:36.287243 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1fb319c0-a1b5-496e-ad70-79a88e971a53 req-6022dfe3-30d1-491d-b469-94cc219ab675 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0410602 May 31 11:58:36.806388 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de3b89d2-ed2b-4ff5-a331-507e2de0328e req-3dfb93bd-e66c-45fa-9267-37fbb3fe89c5 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:36.808098 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-de3b89d2-ed2b-4ff5-a331-507e2de0328e req-3dfb93bd-e66c-45fa-9267-37fbb3fe89c5 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0909538 May 31 11:58:36.993734 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-de3b89d2-ed2b-4ff5-a331-507e2de0328e req-530fb2a3-3a63-40d0-a360-432c61f71793 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1789346 May 31 11:58:37.010510 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-3030db92-72b3-4e1e-8bb2-dfc5c67cba2d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:37.011940 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-3030db92-72b3-4e1e-8bb2-dfc5c67cba2d service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 1098 time: 0.0997498 May 31 11:58:37.156163 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-91552098-bb3c-45b0-8734-e5dcf327b6f8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:37.157632 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-91552098-bb3c-45b0-8734-e5dcf327b6f8 service neutron] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 912 time: 0.1383388 May 31 11:58:37.262037 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-421f3b5f-c963-49a1-a214-7020beeb6103 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=d447eadc-67c7-4eca-8295-e5ae670ce616 HTTP/1.1" status: 200 len: 1001 time: 0.0978215 May 31 11:58:37.329559 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-beb0e592-0320-47e0-8717-502e87baf489 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:37.331242 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-beb0e592-0320-47e0-8717-502e87baf489 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0622547 May 31 11:58:37.406765 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-1d4b8a28-da13-44ee-a6a3-fbd80ef410ce service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:37.408783 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-1d4b8a28-da13-44ee-a6a3-fbd80ef410ce service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0704019 May 31 11:58:37.546874 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-530111a1-7b03-4ca0-ab15-4f58bcad829e service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1310260 May 31 11:58:37.687507 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a011523f-a53b-4c98-92d4-eab448f0e554 req-970b7dba-4d70-4979-9201-1abee45c2538 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1338823 May 31 11:58:38.202542 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c2a4d76-7476-40f2-9656-8a82fd561834 req-7ca052c7-618c-498b-8265-e1f6e926331f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:38.204035 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4c2a4d76-7476-40f2-9656-8a82fd561834 req-7ca052c7-618c-498b-8265-e1f6e926331f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0890677 May 31 11:58:38.255792 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4c2a4d76-7476-40f2-9656-8a82fd561834 req-f39d89e2-2533-444a-85e0-99485cccec1b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0453236 May 31 11:58:38.994758 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:39.480301 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-0f9b142a-95f9-4242-943a-f4386e23df76 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1326442 May 31 11:58:39.491585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d53b0908-5c7f-460d-8dea-afcba717d0c1 req-d614d84e-bc2f-404c-bca3-6a276df576c7 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:39.492959 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d53b0908-5c7f-460d-8dea-afcba717d0c1 req-d614d84e-bc2f-404c-bca3-6a276df576c7 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.1086695 May 31 11:58:39.599986 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d53b0908-5c7f-460d-8dea-afcba717d0c1 req-5b1014d2-5a27-4f99-9ed1-926813a7c20f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0995958 May 31 11:58:39.615834 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-a3cbc8e5-1e9d-46fa-b451-f563aad7ae35 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:39.617189 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-a3cbc8e5-1e9d-46fa-b451-f563aad7ae35 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 1098 time: 0.1150832 May 31 11:58:39.764897 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-827eae54-509f-4a6c-b118-5a22d10385b5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:39.766431 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-827eae54-509f-4a6c-b118-5a22d10385b5 service neutron] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 912 time: 0.1282365 May 31 11:58:39.876099 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-67ee2707-2dd9-4d0d-b18b-8daca63fee85 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=d447eadc-67c7-4eca-8295-e5ae670ce616 HTTP/1.1" status: 200 len: 1001 time: 0.0929356 May 31 11:58:39.943258 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-e10cbca5-2d2b-40b5-822c-3eb31c9e9678 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:39.944721 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-e10cbca5-2d2b-40b5-822c-3eb31c9e9678 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0616381 May 31 11:58:40.031576 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-9f77ba24-82b8-4721-bbb4-bdafc199755c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:40.033071 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-9f77ba24-82b8-4721-bbb4-bdafc199755c service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0779357 May 31 11:58:40.168154 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-1cc70730-f302-42f2-9a2d-7f26a2d0e037 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1251631 May 31 11:58:40.378804 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fc2ea8ca-0b19-48da-b59b-d4f79d95ac3c req-81d1316c-7a77-46d5-bdec-4bc09137cf6e service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2019830 May 31 11:58:40.667022 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35312) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:40.803665 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:40.823756 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:40.827632 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Usage tracker for resource:security_group_rule and tenant:8af1003de5ce49c2869572f705d487be is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:40.830698 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:40.836643 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:40.919189 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:41.006375 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:41.013848 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1728ed31-06bc-494e-b35a-138e1e3ba750 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:41.052680 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d6b69c9a-514a-41fa-b358-1b65ca732a8d,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:41.057073 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=8af1003de5ce49c2869572f705d487be&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3888097 May 31 11:58:41.058024 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:41.059639 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35324) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:41.116923 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-af156ef6-6d6f-4d85-8b44-58e1ba7b9bb0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:41.142311 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-af156ef6-6d6f-4d85-8b44-58e1ba7b9bb0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:41.257868 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-af156ef6-6d6f-4d85-8b44-58e1ba7b9bb0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:41.261960 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4bf91d82-6f4b-47cb-aacf-607fc6794041 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:41.277980 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af156ef6-6d6f-4d85-8b44-58e1ba7b9bb0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d6b69c9a-514a-41fa-b358-1b65ca732a8d,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:41.280875 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-af156ef6-6d6f-4d85-8b44-58e1ba7b9bb0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:41.282827 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-af156ef6-6d6f-4d85-8b44-58e1ba7b9bb0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:41.285492 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-af156ef6-6d6f-4d85-8b44-58e1ba7b9bb0 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/d6b69c9a-514a-41fa-b358-1b65ca732a8d HTTP/1.1" status: 204 len: 168 time: 0.2233934 May 31 11:58:41.989314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35336) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:42.007163 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:42.021841 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:42.025758 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Usage tracker for resource:security_group_rule and tenant:3f417e8fc22a44aab3c1ae4e08363e3d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:42.028496 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:42.034269 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:42.116978 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:42.217485 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:42.226535 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-4c343fec-626a-4a5f-b136-2eb0ad4a078c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:42.268502 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=3f417e8fc22a44aab3c1ae4e08363e3d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2779670 May 31 11:58:42.269442 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9c377faa-896e-4975-88d6-0bcbeee96388,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:42.271707 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35346) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:42.272554 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:42.330499 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3b58b77d-6f92-4e49-a7fc-471bfcfc25c4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:42.355747 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3b58b77d-6f92-4e49-a7fc-471bfcfc25c4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:42.520103 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3b58b77d-6f92-4e49-a7fc-471bfcfc25c4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:42.523804 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bbb7acba-baa9-4128-8ad9-28049d954d25 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:42.535543 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b58b77d-6f92-4e49-a7fc-471bfcfc25c4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9c377faa-896e-4975-88d6-0bcbeee96388,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:42.537231 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3b58b77d-6f92-4e49-a7fc-471bfcfc25c4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.013s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:42.542957 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3b58b77d-6f92-4e49-a7fc-471bfcfc25c4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:42.544216 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-3b58b77d-6f92-4e49-a7fc-471bfcfc25c4 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/9c377faa-896e-4975-88d6-0bcbeee96388 HTTP/1.1" status: 204 len: 168 time: 0.2715130 May 31 11:58:45.787617 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35432) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:45.905549 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:45.919918 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:45.924026 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Usage tracker for resource:security_group_rule and tenant:a428d9ae81d548e9bdb77386c9e96a59 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:45.926822 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:45.932405 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:46.016045 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:46.131308 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:46.139862 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c2441448-6aba-4fb0-bde5-05172ab7d427 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:46.179395 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=a428d9ae81d548e9bdb77386c9e96a59&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3907230 May 31 11:58:46.180965 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=aac3cea3-ecf5-495d-83a8-575eb54c9604,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:46.182165 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35438) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:46.183854 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:46.239562 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-dc687b6a-742c-4754-b117-c223757d83b0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:46.264016 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-dc687b6a-742c-4754-b117-c223757d83b0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:46.403733 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-dc687b6a-742c-4754-b117-c223757d83b0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:46.408945 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6bae2199-ee21-442d-9759-ddc808694fb0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:46.420468 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc687b6a-742c-4754-b117-c223757d83b0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=aac3cea3-ecf5-495d-83a8-575eb54c9604,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:46.423274 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-dc687b6a-742c-4754-b117-c223757d83b0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:46.426299 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-dc687b6a-742c-4754-b117-c223757d83b0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:46.428159 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-dc687b6a-742c-4754-b117-c223757d83b0 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/aac3cea3-ecf5-495d-83a8-575eb54c9604 HTTP/1.1" status: 204 len: 168 time: 0.2448699 May 31 11:58:47.090068 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35462) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:47.107705 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:47.122040 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:47.125853 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Usage tracker for resource:security_group_rule and tenant:98334508a4334caaae010576c1a1b206 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:47.128603 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:47.136466 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:47.217068 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:47.326835 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:47.334634 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f7d7011a-0cc7-4d77-811d-17d5817d3954 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:47.374113 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=98334508a4334caaae010576c1a1b206&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2830091 May 31 11:58:47.375655 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=77b076d0-dd1c-4833-9276-c743aa26e665,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:47.376758 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35470) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:47.378760 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:47.438899 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0ba7fe90-41d6-4e65-852e-a6e5e1ce4d84 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:47.465481 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0ba7fe90-41d6-4e65-852e-a6e5e1ce4d84 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:47.586487 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0ba7fe90-41d6-4e65-852e-a6e5e1ce4d84 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:47.591409 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6946d845-77db-4b04-b323-16ed6be5ea99 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:47.605019 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0ba7fe90-41d6-4e65-852e-a6e5e1ce4d84 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=77b076d0-dd1c-4833-9276-c743aa26e665,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:47.607320 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0ba7fe90-41d6-4e65-852e-a6e5e1ce4d84 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:47.623758 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0ba7fe90-41d6-4e65-852e-a6e5e1ce4d84 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:47.624928 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-0ba7fe90-41d6-4e65-852e-a6e5e1ce4d84 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/77b076d0-dd1c-4833-9276-c743aa26e665 HTTP/1.1" status: 204 len: 168 time: 0.2469041 May 31 11:58:50.909401 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5f5c8f65-c5f3-4378-9970-8d6b7ab04cb0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:50.953942 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35544) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:51.080231 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:51.094727 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:51.099501 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Usage tracker for resource:security_group_rule and tenant:78e733ed2360411f9443e536980cf764 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:51.102299 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:51.108097 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:51.190901 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:51.319884 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:51.329480 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c2070ad2-2608-4d25-9358-39dd102c7a96 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:51.368484 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=78e733ed2360411f9443e536980cf764&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4128754 May 31 11:58:51.370063 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c93900be-e5f4-4c61-a21f-ba0e0b607be0,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:51.371249 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35554) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:51.372948 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:51.438631 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-72912336-70dc-4e0b-a101-fd8d057dae84 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:51.471514 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-72912336-70dc-4e0b-a101-fd8d057dae84 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:51.627527 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-72912336-70dc-4e0b-a101-fd8d057dae84 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:51.632923 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a8b6a313-70c5-4678-9a81-67701cd05e85 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:51.646781 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-72912336-70dc-4e0b-a101-fd8d057dae84 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c93900be-e5f4-4c61-a21f-ba0e0b607be0,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:51.649544 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-72912336-70dc-4e0b-a101-fd8d057dae84 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:51.657453 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-72912336-70dc-4e0b-a101-fd8d057dae84 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:51.661276 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-72912336-70dc-4e0b-a101-fd8d057dae84 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/c93900be-e5f4-4c61-a21f-ba0e0b607be0 HTTP/1.1" status: 204 len: 168 time: 0.2865474 May 31 11:58:52.469594 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35566) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:52.487413 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:52.501545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:58:52.505730 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Usage tracker for resource:security_group_rule and tenant:7a89ba4845c74d858fbf0cb53da1fb50 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:58:52.508709 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:58:52.514405 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:58:52.598391 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:52.697279 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:52.704496 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cb5a7e73-842c-4d90-9ec3-576d68febb10 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:52.750914 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=7a89ba4845c74d858fbf0cb53da1fb50&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2802036 May 31 11:58:52.752932 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e3e0c2c6-b432-4903-b75a-5dfe49104928,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:52.754654 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35574) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:58:52.756449 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:52.812401 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7aab8a0-9361-4aef-a9d4-f81c35a04d0f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:52.838204 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7aab8a0-9361-4aef-a9d4-f81c35a04d0f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:52.971746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7aab8a0-9361-4aef-a9d4-f81c35a04d0f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:52.975363 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-b90e1f40-56a5-4e3a-93a0-375250c9623d tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:52.988139 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7aab8a0-9361-4aef-a9d4-f81c35a04d0f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e3e0c2c6-b432-4903-b75a-5dfe49104928,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:52.991900 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c7aab8a0-9361-4aef-a9d4-f81c35a04d0f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:52.997151 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7aab8a0-9361-4aef-a9d4-f81c35a04d0f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:52.998274 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c7aab8a0-9361-4aef-a9d4-f81c35a04d0f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/e3e0c2c6-b432-4903-b75a-5dfe49104928 HTTP/1.1" status: 204 len: 168 time: 0.2425702 May 31 11:58:53.365851 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:53.481878 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ccf20238-cb97-4242-940a-765c664e646b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:54.611075 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:56.120330 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:56.749345 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-be0c2f98-7bc6-4ca9-87e1-df32aaccbfff req-8c58c0a4-d71b-4926-acac-c24e05570fe5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:56.750815 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-be0c2f98-7bc6-4ca9-87e1-df32aaccbfff req-8c58c0a4-d71b-4926-acac-c24e05570fe5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 946 time: 0.0796573 May 31 11:58:56.796709 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-be0c2f98-7bc6-4ca9-87e1-df32aaccbfff req-15dd9a76-e7aa-4ef2-8ea8-c27b79ead3fd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393782 May 31 11:58:58.002010 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-626c8bf6-2330-4592-ac69-81a484daf874 req-1e4d9157-3f24-43ce-a0f8-3d5f3a612820 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:58.003908 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-626c8bf6-2330-4592-ac69-81a484daf874 req-1e4d9157-3f24-43ce-a0f8-3d5f3a612820 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 946 time: 0.0904014 May 31 11:58:58.049142 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-626c8bf6-2330-4592-ac69-81a484daf874 req-4da1f753-5582-43d8-b4c0-b69eb5dd7a44 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=8e2d5996-59e7-4c4a-9bfd-8eee9c767259&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389194 May 31 11:58:58.336470 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68f38e00-905d-454c-a5a4-554e63ce410a req-83d3b6f9-9374-4663-9090-a9a339c694b0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:58:58.337871 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-68f38e00-905d-454c-a5a4-554e63ce410a req-83d3b6f9-9374-4663-9090-a9a339c694b0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 946 time: 0.0826883 May 31 11:58:58.470985 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Deleting port e0a8cb19-42fc-4a14-b86a-efda3d732b98 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:58:58.471306 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:58.472571 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:58:58.604522 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] For port e0a8cb19-42fc-4a14-b86a-efda3d732b98, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e0a8cb19-42fc-4a14-b86a-efda3d732b98,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:58:58.604875 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:58.898286 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:58.898650 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:58.899080 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Floating IP abe27c62-2687-4502-83b6-bd3a48c06f27 disassociated. External IP: 172.27.1.172, port: e0a8cb19-42fc-4a14-b86a-efda3d732b98. May 31 11:58:58.899411 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Calling delete_port for e0a8cb19-42fc-4a14-b86a-efda3d732b98 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:58:59.125461 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:59.131462 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:58:59.288104 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event deleted for resources: {'Port': ['ID=e0a8cb19-42fc-4a14-b86a-efda3d732b98,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:58:59.300004 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.169s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:58:59.303525 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-90a8bf14-a09e-4133-aeac-668a4fb4d472 req-abe15fe8-effb-4c0a-9900-16bf7ba2b57d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 210 time: 0.1378653 May 31 11:58:59.488731 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:58:59.674830 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:58:59.685216 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] Sync routers for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:58:59.696047 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:58:59.696681 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:58:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:58:59.752125 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] No DVR routers for this DVR port e0a8cb19-42fc-4a14-b86a-efda3d732b98 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:58:59.754704 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:58:59.757601 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-deleted', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:58:59.759913 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-0780e4ca-117a-46c9-af24-c0fdd4ae65ec" -d '{"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-deleted", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:58:59.805780 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:58:59 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-cfd47655-655c-4099-9bb0-a4acab810c0d x-openstack-request-id: req-cfd47655-655c-4099-9bb0-a4acab810c0d {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:58:59.806041 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "0fa22058-5033-48f3-9ade-9796c8b05c17", "name": "network-vif-deleted", "tag": "e0a8cb19-42fc-4a14-b86a-efda3d732b98", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:58:59.806317 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-cfd47655-655c-4099-9bb0-a4acab810c0d {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:58:59.806871 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0fa22058-5033-48f3-9ade-9796c8b05c17', 'name': 'network-vif-deleted', 'tag': 'e0a8cb19-42fc-4a14-b86a-efda3d732b98', 'status': 'completed', 'code': 200} May 31 11:58:59.874690 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-68f38e00-905d-454c-a5a4-554e63ce410a req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "DELETE /v2.0/ports/e0a8cb19-42fc-4a14-b86a-efda3d732b98 HTTP/1.1" status: 204 len: 173 time: 1.5311987 May 31 11:58:59.885342 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:00.240007 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:00.244621 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] Checking router: 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:00.254316 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:00.254932 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:58:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:00.308060 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35724) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:00.464365 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:00.481866 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:00.487064 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Usage tracker for resource:security_group_rule and tenant:087a01a768a6404dbf662ec0f1f7c023 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:00.490888 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:00.492347 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:00.497632 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:00.510203 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-0d818c34-7459-4d2d-9418-e4ff209065a7 req-b7b5dc70-aacf-482e-9081-92c58c81e8ba tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Synchronizing usage tracker for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:00.607869 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:00.630501 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35734) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:00.648964 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-0d818c34-7459-4d2d-9418-e4ff209065a7 req-b7b5dc70-aacf-482e-9081-92c58c81e8ba tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Unset dirty status for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:00.650178 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0d818c34-7459-4d2d-9418-e4ff209065a7 req-b7b5dc70-aacf-482e-9081-92c58c81e8ba tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=0fa22058-5033-48f3-9ade-9796c8b05c17 HTTP/1.1" status: 200 len: 210 time: 0.1937227 May 31 11:59:00.683330 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:00.710713 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7504997a-9f94-4665-a9e1-22228306a393 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:00.718946 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=087a01a768a6404dbf662ec0f1f7c023&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4092786 May 31 11:59:00.722695 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35740) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:00.741465 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5f7c5aa5-1de7-4883-b8f5-2434eb49ca3a,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:00.745897 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:00.757527 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:00.801538 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: e0a8cb19-42fc-4a14-b86a-efda3d732b98 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:00.802220 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e0a8cb19-42fc-4a14-b86a-efda3d732b98 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:00.809416 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3c3fe967-aa76-4a5e-a9a2-57a57e4a10a2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:00.811541 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:00.828170 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:00.833653 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Usage tracker for resource:security_group_rule and tenant:7bd609bd61ce47279c3633dfee92a7cf is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:00.836723 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:00.836867 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3c3fe967-aa76-4a5e-a9a2-57a57e4a10a2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:00.842979 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:00.920258 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:00.926107 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-34967797-7014-4ae7-8b98-dad48f5583ca None None] The sync data for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi is: [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:00.932956 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:00.980930 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3c3fe967-aa76-4a5e-a9a2-57a57e4a10a2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:00.986483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-225c1b8a-22d1-4a23-ae12-fe0b65a23e00 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:00.999632 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3c3fe967-aa76-4a5e-a9a2-57a57e4a10a2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5f7c5aa5-1de7-4883-b8f5-2434eb49ca3a,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:01.002813 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3c3fe967-aa76-4a5e-a9a2-57a57e4a10a2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:01.005448 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3c3fe967-aa76-4a5e-a9a2-57a57e4a10a2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:01.006855 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-3c3fe967-aa76-4a5e-a9a2-57a57e4a10a2 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/5f7c5aa5-1de7-4883-b8f5-2434eb49ca3a HTTP/1.1" status: 204 len: 168 time: 0.2831755 May 31 11:59:01.057983 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:01.067790 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:01.117866 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35748) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:01.118070 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=7bd609bd61ce47279c3633dfee92a7cf&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4862428 May 31 11:59:01.119747 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=520a209b-bfc4-499d-938e-5bb79b39b549,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:01.123014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:01.206819 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0425d46f-134c-4374-8b45-1b41d98e794b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:01.217698 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-99ef00ce-383d-40f0-a812-5f4f2e0d3e8c None None] New status for floating IP abe27c62-2687-4502-83b6-bd3a48c06f27: DOWN {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 11:59:01.234602 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0425d46f-134c-4374-8b45-1b41d98e794b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:01.360043 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0425d46f-134c-4374-8b45-1b41d98e794b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:01.367444 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f292e408-c3e7-4dc3-acb0-97d34c69d088 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:01.383988 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0425d46f-134c-4374-8b45-1b41d98e794b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=520a209b-bfc4-499d-938e-5bb79b39b549,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:01.389830 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0425d46f-134c-4374-8b45-1b41d98e794b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:01.408397 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0425d46f-134c-4374-8b45-1b41d98e794b admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:01.416229 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-0425d46f-134c-4374-8b45-1b41d98e794b admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/520a209b-bfc4-499d-938e-5bb79b39b549 HTTP/1.1" status: 204 len: 168 time: 0.2963421 May 31 11:59:01.896197 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35764) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:01.904581 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35768) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:01.915669 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:01.930946 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:01.935089 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Usage tracker for resource:security_group_rule and tenant:79bc4036dd214dcf83689b535a239dce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:01.938145 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:01.944481 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:01.977205 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ef6aede4-c084-45ad-bd9b-ffb1b7219573 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.075524 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ef6aede4-c084-45ad-bd9b-ffb1b7219573 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.203703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.228172 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ef6aede4-c084-45ad-bd9b-ffb1b7219573 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.233747 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-9464a2e0-4085-4676-ac22-de71ec93cb35 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:02.253523 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ef6aede4-c084-45ad-bd9b-ffb1b7219573 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=8e2d5996-59e7-4c4a-9bfd-8eee9c767259,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:02.256665 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-ef6aede4-c084-45ad-bd9b-ffb1b7219573 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:02.297064 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.305735 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4714ffb4-98fd-4acd-bb54-3cb391360752 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:02.340033 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ef6aede4-c084-45ad-bd9b-ffb1b7219573 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.341145 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-ef6aede4-c084-45ad-bd9b-ffb1b7219573 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "DELETE /v2.0/security-groups/8e2d5996-59e7-4c4a-9bfd-8eee9c767259 HTTP/1.1" status: 204 len: 168 time: 0.4349213 May 31 11:59:02.343953 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35780) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:02.345497 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=79bc4036dd214dcf83689b535a239dce&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4466865 May 31 11:59:02.345866 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=cf30fe0c-53ba-475c-8aba-f66217fdb6cf,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:02.348154 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35784) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:02.361098 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:02.361897 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.api.v2.resource [None req-ab4f921c-4e13-45b4-be9f-49bf63816695 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] show failed (client error): The resource could not be found. May 31 11:59:02.362877 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-ab4f921c-4e13-45b4-be9f-49bf63816695 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups/8e2d5996-59e7-4c4a-9bfd-8eee9c767259 HTTP/1.1" status: 404 len: 347 time: 0.0180337 May 31 11:59:02.365756 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35788) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:02.417130 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a791886d-0702-4e0e-81cd-d48288d1e988 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.444310 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a791886d-0702-4e0e-81cd-d48288d1e988 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.594234 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a791886d-0702-4e0e-81cd-d48288d1e988 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.599645 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-af156ef6-6d6f-4d85-8b44-58e1ba7b9bb0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:02.607277 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Deleting port e86adcce-af56-4a3c-8c81-4014ee2a0b72 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:02.607681 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.609818 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a791886d-0702-4e0e-81cd-d48288d1e988 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=cf30fe0c-53ba-475c-8aba-f66217fdb6cf,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:02.613969 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a791886d-0702-4e0e-81cd-d48288d1e988 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:02.617358 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a791886d-0702-4e0e-81cd-d48288d1e988 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.621064 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a791886d-0702-4e0e-81cd-d48288d1e988 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/cf30fe0c-53ba-475c-8aba-f66217fdb6cf HTTP/1.1" status: 204 len: 168 time: 0.2694957 May 31 11:59:02.727596 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.727919 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:02.866296 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Calling delete_port for e86adcce-af56-4a3c-8c81-4014ee2a0b72 owned by network:floatingip {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:03.078079 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:03.082397 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-96d7aa3f-8285-4746-8081-c30f9f16b90b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:03.132042 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event deleted for resources: {'Port': ['ID=e86adcce-af56-4a3c-8c81-4014ee2a0b72,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:03.135294 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:03.236834 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:03.325849 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:03.327004 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "DELETE /v2.0/floatingips/abe27c62-2687-4502-83b6-bd3a48c06f27 HTTP/1.1" status: 204 len: 168 time: 0.9602852 May 31 11:59:04.700290 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35858) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:04.705382 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'security_group': {'name': 'tempest-securitygroup--492617665', 'description': 'tempest-description--1053833199'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:04.711269 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:04.716018 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:security_group and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:04.718265 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:04.725620 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:04.816793 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:04.837734 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:04.841505 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:security_group_rule and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:04.844340 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:04.850834 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:04.920564 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.028347 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.034235 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c7aab8a0-9361-4aef-a9d4-f81c35a04d0f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:05.075155 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'SecurityGroup': ['ID=f3375ede-3e88-45fd-8179-4673610130ba,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:05.077788 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:05.278401 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.279572 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 0.5792708 May 31 11:59:05.282374 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35870) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:05.287305 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'security_group_rule': {'security_group_id': 'f3375ede-3e88-45fd-8179-4673610130ba', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:05.293258 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:05.297225 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:security_group_rule and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:05.300673 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:05.306455 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:05.508107 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.570973 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.712242 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.714516 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-647fe9ab-aa93-49e6-8be3-9f4d3678e53a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:05.719439 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35880) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:05.738176 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=aacc31c1-d2ee-475a-b7b0-c98ebf7d06e7,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:05.740865 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:05.805463 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.806730 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f3b84c03-e516-43da-b725-3e6b0b2f57f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.5234511 May 31 11:59:05.809853 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35888) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:05.815308 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'security_group_rule': {'security_group_id': 'f3375ede-3e88-45fd-8179-4673610130ba', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:05.822309 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:05.826105 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:security_group_rule and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:05.828947 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:05.847925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:05.855299 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.876814 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:05.880936 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Usage tracker for resource:security_group_rule and tenant:bc6d592431c347038f98c1028b73a7af is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:05.884209 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:05.890388 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:05.975031 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:05.976462 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.039648 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.056551 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.064647 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a791886d-0702-4e0e-81cd-d48288d1e988 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:06.108291 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=bc6d592431c347038f98c1028b73a7af&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3865323 May 31 11:59:06.109196 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7ec870bd-3791-4cd6-8172-1ae05a68afaf,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:06.111489 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35894) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:06.112605 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:06.169966 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0e916296-54d4-47e4-88b9-3e14fec4f8d1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.182444 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.184475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:06.195621 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0e916296-54d4-47e4-88b9-3e14fec4f8d1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.207955 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=3b49453a-cd88-471e-bd26-f9f12f7e63aa,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:06.213432 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:06.328980 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.330801 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.5198739 May 31 11:59:06.333903 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35898) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:06.338976 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:06.346108 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:06.350048 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Usage tracker for resource:floatingip and tenant:0e7f0a333ee54b31bcd5c2290c3e284d is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:06.352904 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:06.358678 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:06.369247 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0e916296-54d4-47e4-88b9-3e14fec4f8d1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.374093 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d904f782-979d-4619-922b-6b9e343c46a3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:06.387237 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e916296-54d4-47e4-88b9-3e14fec4f8d1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7ec870bd-3791-4cd6-8172-1ae05a68afaf,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:06.390002 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0e916296-54d4-47e4-88b9-3e14fec4f8d1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:06.390928 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0e916296-54d4-47e4-88b9-3e14fec4f8d1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.393310 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-0e916296-54d4-47e4-88b9-3e14fec4f8d1 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/7ec870bd-3791-4cd6-8172-1ae05a68afaf HTTP/1.1" status: 204 len: 168 time: 0.2796533 May 31 11:59:06.466598 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.608063 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:06.752344 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Allocated IP 172.27.1.9 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/56020899-a5d2-4887-8fc3-6fe9b3416c61) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:06.903428 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.055331 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.060055 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-36849b09-04d3-496d-a12e-1dc3a51b4e63 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:07.213478 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:07.216617 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:07.220859 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:07.230993 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35914) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:07.248309 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.250938 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'Port': ['ID=56020899-a5d2-4887-8fc3-6fe9b3416c61,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:07.258063 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:07.265318 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:07.276873 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Usage tracker for resource:security_group_rule and tenant:18bf20995e254e39bd4d176f841dc42b is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:07.276873 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:07.279961 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:07.308817 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.365659 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.489793 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.506845 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.515589 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-75b37657-8865-4ed1-8297-5919c528be35 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:07.560435 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=18bf20995e254e39bd4d176f841dc42b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3288300 May 31 11:59:07.560435 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b11a2f78-20c1-41ea-9cc7-fa395913f01f,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:07.562241 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35920) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:07.576953 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:07.635885 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d0060474-efc6-423b-93b8-df1580c568d1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.659109 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35924) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:07.668705 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d0060474-efc6-423b-93b8-df1580c568d1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.709203 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.786847 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.793937 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.796832 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d0060474-efc6-423b-93b8-df1580c568d1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.799518 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e14f4eeb-9d46-4b54-96a3-e77122efd0e4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:07.802654 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7cb52ae7-fc56-4060-957a-e5f0f2ab66a6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:07.817080 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d0060474-efc6-423b-93b8-df1580c568d1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b11a2f78-20c1-41ea-9cc7-fa395913f01f,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:07.819496 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d0060474-efc6-423b-93b8-df1580c568d1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:07.820575 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:07.827583 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Usage tracker for resource:security_group_rule and tenant:858c29d7bd1f420da52bc8cdf41ead9e is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:07.831051 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:07.836700 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d0060474-efc6-423b-93b8-df1580c568d1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.840454 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d0060474-efc6-423b-93b8-df1580c568d1 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/b11a2f78-20c1-41ea-9cc7-fa395913f01f HTTP/1.1" status: 204 len: 168 time: 0.2644873 May 31 11:59:07.840454 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:07.906173 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'Port': ['ID=56020899-a5d2-4887-8fc3-6fe9b3416c61,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:07.911054 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:07.915855 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:07.931188 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:08.102469 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:08.102657 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:08.103707 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 691 time: 1.7689521 May 31 11:59:08.111165 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-23731588-705f-4d27-84d7-a0816df922a6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:08.153150 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=858c29d7bd1f420da52bc8cdf41ead9e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4930205 May 31 11:59:08.154088 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d4cfafe8-418e-4f9f-a41c-71ce04244f46,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:08.155908 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35934) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:08.157066 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:08.220844 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c552f173-22be-49e3-9245-45533f584ee7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:08.247678 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c552f173-22be-49e3-9245-45533f584ee7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:08.334437 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1aab7df3-ceaa-4e06-ab6d-50e6adb7e51a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-securitygroup--492617665&fields=id&tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d HTTP/1.1" status: 200 len: 267 time: 0.0425136 May 31 11:59:08.434562 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c552f173-22be-49e3-9245-45533f584ee7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:08.442556 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0425d46f-134c-4374-8b45-1b41d98e794b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:08.457161 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c552f173-22be-49e3-9245-45533f584ee7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d4cfafe8-418e-4f9f-a41c-71ce04244f46,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:08.460529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c552f173-22be-49e3-9245-45533f584ee7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:08.467783 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c552f173-22be-49e3-9245-45533f584ee7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:08.471408 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c552f173-22be-49e3-9245-45533f584ee7 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/d4cfafe8-418e-4f9f-a41c-71ce04244f46 HTTP/1.1" status: 204 len: 168 time: 0.3120244 May 31 11:59:08.519689 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-16f500b1-c281-421a-84da-da047635caca tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:08.525417 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-16f500b1-c281-421a-84da-da047635caca tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 816 time: 0.1799920 May 31 11:59:08.537198 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-efa20014-e6f3-4f7f-8c00-558b62cfe2bc tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/quotas/0e7f0a333ee54b31bcd5c2290c3e284d HTTP/1.1" status: 200 len: 375 time: 0.0098307 May 31 11:59:08.624456 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-666deccc-7311-408f-a743-a70aac570f8b req-467eec7e-5ca5-4b45-a166-ee00fa603376 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-104550288&fields=id&tenant_id=39ae37364a184ae7ae341ea7b2847c35 HTTP/1.1" status: 200 len: 267 time: 0.0428445 May 31 11:59:08.633111 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-bc9fa4f1-9e06-45bf-8d09-caffec56b7da tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0901046 May 31 11:59:08.726792 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-666deccc-7311-408f-a743-a70aac570f8b req-c263e12e-7baa-4647-aff5-bc9fee48f8c3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:08.728160 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-666deccc-7311-408f-a743-a70aac570f8b req-c263e12e-7baa-4647-aff5-bc9fee48f8c3 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 984 time: 0.0970786 May 31 11:59:08.742001 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [req-666deccc-7311-408f-a743-a70aac570f8b req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Request body: {'port': {'security_groups': ['7392633c-3cf7-4366-bb3e-fd78503c0a30']}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:08.789183 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-56f5c904-0961-44fc-bdd1-e70249b7d1fe service neutron] 10.222.0.48 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1434133 May 31 11:59:08.874916 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-666deccc-7311-408f-a743-a70aac570f8b req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:08.946083 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-e01cbb4d-ec4d-4fa4-971e-97bec74bb79b service neutron] 10.222.0.48 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1503260 May 31 11:59:09.074174 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.215348 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35976) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:09.244906 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-666deccc-7311-408f-a743-a70aac570f8b req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] For port 3c46c8f8-777e-4ff2-8577-976c595eec9e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:09.273945 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.275384 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-666deccc-7311-408f-a743-a70aac570f8b req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.297555 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1e13d175-629d-4779-8677-9d4eb0f33160 req-d1350fd4-4f45-44fb-8f10-3661d5e54c36 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 210 time: 0.0619810 May 31 11:59:09.300308 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:09.304243 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Usage tracker for resource:security_group_rule and tenant:7b8014071844402b915cc2dcb32781d8 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:09.307132 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:09.312921 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:09.354803 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-666deccc-7311-408f-a743-a70aac570f8b req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.365338 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-03dce496-54e0-48f9-b8f8-daac41866ad5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:09.367050 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['3c46c8f8-777e-4ff2-8577-976c595eec9e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:59:09.448937 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-666deccc-7311-408f-a743-a70aac570f8b req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:09.449590 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-666deccc-7311-408f-a743-a70aac570f8b req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.450954 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-666deccc-7311-408f-a743-a70aac570f8b req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "PUT /v2.0/ports/3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 942 time: 0.7144375 May 31 11:59:09.451458 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:09.458502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc" -d '{"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:09.458652 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 35986) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:09.477133 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.477767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event updated for resources: {'Port': ['ID=3c46c8f8-777e-4ff2-8577-976c595eec9e,revision_number=7']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:09.481044 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:09.531287 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:59:09 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-e9feb42e-dc15-49cb-9729-59417ea8219a x-openstack-request-id: req-e9feb42e-dc15-49cb-9729-59417ea8219a {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:09.531870 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:09.532864 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-e9feb42e-dc15-49cb-9729-59417ea8219a {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:09.533463 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'status': 'completed', 'code': 200} May 31 11:59:09.544772 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['3c46c8f8-777e-4ff2-8577-976c595eec9e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:59:09.614652 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9dd4baa8-eeee-46e6-a87e-592493810a8e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.650014 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.669679 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-27aaf842-c18f-4d6a-a0ab-6864d7e01358 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:09.699795 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9dd4baa8-eeee-46e6-a87e-592493810a8e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.760571 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-778c51b6-951a-4f76-bdd4-262f39dae3dd None None] Provisioning complete for port 3c46c8f8-777e-4ff2-8577-976c595eec9e triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:09.760947 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-778c51b6-951a-4f76-bdd4-262f39dae3dd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.784145 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=7b8014071844402b915cc2dcb32781d8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5677049 May 31 11:59:09.786047 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=745e808f-7b29-4777-8116-c9b3fa573440,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:09.787013 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 35994) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:09.791327 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:09.795030 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-51db51a6-a3ac-4660-a39f-248ef195352e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:09.796485 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-51db51a6-a3ac-4660-a39f-248ef195352e service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=39ae37364a184ae7ae341ea7b2847c35&device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1098 time: 0.2364028 May 31 11:59:09.866862 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-096cc375-132f-4801-ac17-0e2a9b6a91ac admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.905178 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-096cc375-132f-4801-ac17-0e2a9b6a91ac admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:09.997600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9dd4baa8-eeee-46e6-a87e-592493810a8e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:10.004833 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4c51b31a-6ccf-4858-8536-e7153000532c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:10.027427 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9dd4baa8-eeee-46e6-a87e-592493810a8e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=82252074-5d18-415f-aaff-3f993aa67988,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:10.042651 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9dd4baa8-eeee-46e6-a87e-592493810a8e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:10.064446 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-8da7f7d4-dde0-4f98-a2dc-4c0ae7cd44fa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:10.065952 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-8da7f7d4-dde0-4f98-a2dc-4c0ae7cd44fa service neutron] 10.222.0.47 "GET /v2.0/networks?id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb HTTP/1.1" status: 200 len: 915 time: 0.2624190 May 31 11:59:10.066195 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-096cc375-132f-4801-ac17-0e2a9b6a91ac admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:10.071053 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-dd5be150-260c-4c01-9575-7154d9a0f310 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:10.084002 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-096cc375-132f-4801-ac17-0e2a9b6a91ac admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=745e808f-7b29-4777-8116-c9b3fa573440,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:10.087836 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-096cc375-132f-4801-ac17-0e2a9b6a91ac admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:10.093198 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-096cc375-132f-4801-ac17-0e2a9b6a91ac admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:10.094422 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-096cc375-132f-4801-ac17-0e2a9b6a91ac admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/745e808f-7b29-4777-8116-c9b3fa573440 HTTP/1.1" status: 204 len: 168 time: 0.3063767 May 31 11:59:10.179114 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9dd4baa8-eeee-46e6-a87e-592493810a8e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:10.179114 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-9dd4baa8-eeee-46e6-a87e-592493810a8e tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "DELETE /v2.0/security-groups/82252074-5d18-415f-aaff-3f993aa67988 HTTP/1.1" status: 204 len: 168 time: 0.7160287 May 31 11:59:10.179114 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36008) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:10.254227 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-0dcce8e4-53ea-480e-b69f-14ea54e425a3 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 1001 time: 0.1814466 May 31 11:59:10.406106 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-94303281-0faf-4898-b829-eb6c5de8cc7f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:10.408088 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-94303281-0faf-4898-b829-eb6c5de8cc7f service neutron] 10.222.0.47 "GET /v2.0/subnets?id=672ec900-b55c-44a0-9ed9-7e27beeace12 HTTP/1.1" status: 200 len: 853 time: 0.1469717 May 31 11:59:10.469247 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-15364c53-6f41-4c2c-8fee-fe221d419e54 req-aaf8f1d2-d0b3-47cf-a613-ce52b2a16347 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Synchronizing usage tracker for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:10.578865 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-501f7fa7-1453-4a71-a333-3e59b62ad087 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:10.580899 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-501f7fa7-1453-4a71-a333-3e59b62ad087 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1651676 May 31 11:59:10.582267 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-15364c53-6f41-4c2c-8fee-fe221d419e54 req-aaf8f1d2-d0b3-47cf-a613-ce52b2a16347 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Unset dirty status for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:10.584295 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-15364c53-6f41-4c2c-8fee-fe221d419e54 req-aaf8f1d2-d0b3-47cf-a613-ce52b2a16347 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 210 time: 0.1627765 May 31 11:59:10.611859 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Deleting port 834ff228-3f9e-460f-96e4-cd4df6eec21b {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:10.612276 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:10.824268 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:10.824610 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:10.833914 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-d2d69ef2-33f3-4396-a192-d3ff280cd2bf service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2431040 May 31 11:59:10.970395 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Calling delete_port for 834ff228-3f9e-460f-96e4-cd4df6eec21b owned by network:floatingip {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:11.109580 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2aca82d9-cdb0-4f91-b2ee-a9971cb35ffc req-64da129c-7336-4e9a-a61d-15e6610dc4e9 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2218659 May 31 11:59:11.314664 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:11.319489 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-55750ac5-76d9-4d70-8cce-f90d411928c2 tempest-VolumesNegativeTest-645459826 tempest-VolumesNegativeTest-645459826-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:11.365698 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event deleted for resources: {'Port': ['ID=834ff228-3f9e-460f-96e4-cd4df6eec21b,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:11.369601 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:11.459603 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks [] for floatingip, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:11.459998 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Floating IP 7e57ce71-e2b7-4a92-a07c-796d6beacdf2 disassociated (deleted). External IP: 172.27.1.199, port: 3c46c8f8-777e-4ff2-8577-976c595eec9e. May 31 11:59:11.780225 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9c2f5ba6-98e5-45a1-a1fa-66b0e8eb0dc5 req-a30e6961-b5fd-4b58-b7dc-1326efa26e4a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 210 time: 0.0632725 May 31 11:59:12.094268 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Router d195a538-22a1-483f-a3e5-9007c6338ece has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:59:12.190047 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-73a44e64-232e-49d4-80d1-8c1646c4e35e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:12.191430 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-73a44e64-232e-49d4-80d1-8c1646c4e35e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 816 time: 0.2521234 May 31 11:59:12.321674 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-ca19fb77-00d7-41d0-8761-b2db1d156a91 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d HTTP/1.1" status: 200 len: 379 time: 0.1146276 May 31 11:59:12.336699 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'port': {'device_id': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'admin_state_up': True, 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'security_groups': ['f3375ede-3e88-45fd-8179-4673610130ba']}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:12.437392 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:12.447850 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] Sync routers for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:12.457282 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:12.458082 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:58:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:12.499523 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:12.510219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:12.576077 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:12.617935 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Made reservation on behalf of 0e7f0a333ee54b31bcd5c2290c3e284d for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:12.621349 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:12.644992 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:12.810411 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "DELETE /v2.0/floatingips/7e57ce71-e2b7-4a92-a07c-796d6beacdf2 HTTP/1.1" status: 204 len: 168 time: 2.6306093 May 31 11:59:12.833645 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:12.846643 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1" -d '{"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:12.866332 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Allocated IP 172.20.0.7 (19c09cdd-9420-4946-a105-024d5ea86ec3/4c55d678-c52f-4b1e-b005-86edad0fbf12/24163e07-db8d-4a12-8f62-285a1abb77fa) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:12.967464 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c1f3a97f-5b6f-47aa-8875-846ba6f267c7 req-d4ded6d4-dce2-4af6-a266-3a1a00461568 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 210 time: 0.0575764 May 31 11:59:13.083443 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:59:12 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-aba6b15c-4cd3-48f8-9660-9820ef7d9396 x-openstack-request-id: req-aba6b15c-4cd3-48f8-9660-9820ef7d9396 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:13.086718 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:13.087002 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-aba6b15c-4cd3-48f8-9660-9820ef7d9396 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:13.087437 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'status': 'completed', 'code': 200} May 31 11:59:13.097666 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:11Z', 'revision_number': 5, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:13.103032 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] Checking router: d195a538-22a1-483f-a3e5-9007c6338ece for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:13.112592 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:13.113584 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:58:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:13.174767 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:13.210298 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Transition to ACTIVE for port object 24163e07-db8d-4a12-8f62-285a1abb77fa will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:13.221316 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-6d41536a-5d8f-40da-a77d-5c5d820e22c0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:13.223343 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-6d41536a-5d8f-40da-a77d-5c5d820e22c0 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=39ae37364a184ae7ae341ea7b2847c35&device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 1098 time: 0.1273096 May 31 11:59:13.343788 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:13.366550 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:13.372655 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:13.379257 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-e7311165-4283-4a46-98aa-a45316e2b839 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:13.379257 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-e7311165-4283-4a46-98aa-a45316e2b839 service neutron] 10.222.0.47 "GET /v2.0/networks?id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb HTTP/1.1" status: 200 len: 915 time: 0.1446216 May 31 11:59:13.440167 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:13.451765 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-c7240cde-b339-41ea-ad44-74a74963fbcd service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 200 len: 217 time: 0.0649047 May 31 11:59:13.452076 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:13.456999 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:13.532721 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:13.537367 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.165s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:13.556031 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:13.556701 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:13.562886 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-22869a88-61a4-440c-be16-4995a09b6aed service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:13.565614 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-22869a88-61a4-440c-be16-4995a09b6aed service neutron] 10.222.0.47 "GET /v2.0/subnets?id=672ec900-b55c-44a0-9ed9-7e27beeace12 HTTP/1.1" status: 200 len: 853 time: 0.0961332 May 31 11:59:13.662059 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-a2b9b716-887f-4a44-b05d-baa987023aca service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:13.663542 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-a2b9b716-887f-4a44-b05d-baa987023aca service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=0fae71a7-8e60-4f7d-a467-822e3d8d87fb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0881882 May 31 11:59:13.680153 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.3492291 May 31 11:59:13.693554 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Request body: {'port': {'device_id': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:13.803970 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:11Z', 'revision_number': 5, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:13.808290 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ffeaafb7-2b36-4703-a6ed-f629396b7c04 None None] The sync data for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi is: [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:56:04Z', 'revision_number': 4, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '6b60438b-78d8-4039-a0ce-c42d04559f08', 'name': '', 'network_id': '0fae71a7-8e60-4f7d-a467-822e3d8d87fb', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'mac_address': 'fa:16:3e:a4:d7:48', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_interface', 'standard_attr_id': 209, 'fixed_ips': [{'subnet_id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:03Z', 'updated_at': '2021-05-31T11:56:11Z', 'revision_number': 5, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'subnets': [{'id': '672ec900-b55c-44a0-9ed9-7e27beeace12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:13.823557 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-9f59fd8b-0e07-4e5c-842c-49abced1c13d service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1524475 May 31 11:59:13.827750 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-29ba5e7d-3602-4937-b563-1ef614c4400e None None] Provisioning for port 24163e07-db8d-4a12-8f62-285a1abb77fa completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:13.830314 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:13.833763 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-29ba5e7d-3602-4937-b563-1ef614c4400e None None] Provisioning complete for port 24163e07-db8d-4a12-8f62-285a1abb77fa triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:13.834210 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-29ba5e7d-3602-4937-b563-1ef614c4400e None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:13.928890 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-29ba5e7d-3602-4937-b563-1ef614c4400e None None] Port 24163e07-db8d-4a12-8f62-285a1abb77fa cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:59:13.999596 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c6bb3fc9-84ad-441a-9e7f-8d4d1e6c50f1 req-6df947c3-ce30-4c03-ba42-4f92870fc772 service neutron] 10.222.0.47 "GET /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1646290 May 31 11:59:14.230903 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f72237b5-dfd9-4f63-a6cb-420b6aa0ce35 None None] New status for floating IP 7e57ce71-e2b7-4a92-a07c-796d6beacdf2: DOWN {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 11:59:14.372954 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-550879ff-1280-4fb8-9c9f-c0bad77f8387 req-88ee9f1d-a4ae-42b8-8b62-789414cc6940 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:14.373552 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-550879ff-1280-4fb8-9c9f-c0bad77f8387 req-88ee9f1d-a4ae-42b8-8b62-789414cc6940 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Synchronizing usage tracker for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:14.478384 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 24163e07-db8d-4a12-8f62-285a1abb77fa {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:59:14.483037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-550879ff-1280-4fb8-9c9f-c0bad77f8387 req-88ee9f1d-a4ae-42b8-8b62-789414cc6940 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Unset dirty status for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:14.484176 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-550879ff-1280-4fb8-9c9f-c0bad77f8387 req-88ee9f1d-a4ae-42b8-8b62-789414cc6940 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 931 time: 0.3210616 May 31 11:59:14.488750 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:14.523116 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Transition to ACTIVE for port object 24163e07-db8d-4a12-8f62-285a1abb77fa will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:14.541948 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-550879ff-1280-4fb8-9c9f-c0bad77f8387 req-b9795169-554d-436f-8179-e28cdc4cee47 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0506387 May 31 11:59:14.602407 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:14.603179 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:14.736691 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:14.776085 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.173s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:14.863142 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Attempting to bind port 24163e07-db8d-4a12-8f62-285a1abb77fa on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:59:14.863663 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Attempting to bind port 24163e07-db8d-4a12-8f62-285a1abb77fa by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:59:14.864191 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Attempting to bind port 24163e07-db8d-4a12-8f62-285a1abb77fa on network 19c09cdd-9420-4946-a105-024d5ea86ec3 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:59:14.869527 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Port 24163e07-db8d-4a12-8f62-285a1abb77fa on network 19c09cdd-9420-4946-a105-024d5ea86ec3 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:59:14.869895 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Attempting to bind port 24163e07-db8d-4a12-8f62-285a1abb77fa on network 19c09cdd-9420-4946-a105-024d5ea86ec3 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:59:14.875228 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 59, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:59:14.875577 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Checking segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:14.876655 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Bound using segment: {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:59:14.877209 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Bound port: 24163e07-db8d-4a12-8f62-285a1abb77fa, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '15b69bd1-9fea-43fa-badd-39be1ac15eb5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 371, 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:59:14.877583 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:14.978625 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:59:14.981790 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:59:15.246028 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:15.254848 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:15.305792 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:15.306402 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:15.307744 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] 10.222.0.47 "PUT /v2.0/ports/24163e07-db8d-4a12-8f62-285a1abb77fa HTTP/1.1" status: 200 len: 1093 time: 1.6199582 May 31 11:59:15.308223 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:15.310480 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-2475161c-cb5a-4e69-b143-59ca39a0d778" -d '{"events": [{"name": "network-changed", "server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:15.343159 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:15.350396 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:15.361489 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:59:15 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-e7a882a6-44fe-4f98-8a13-2dbc22ae1f6c x-openstack-request-id: req-e7a882a6-44fe-4f98-8a13-2dbc22ae1f6c {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:15.361882 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:15.362165 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-e7a882a6-44fe-4f98-8a13-2dbc22ae1f6c {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:15.362589 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'status': 'completed', 'code': 200} May 31 11:59:15.474348 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2475161c-cb5a-4e69-b143-59ca39a0d778 req-0c5727b0-ac5a-4916-bd27-af00eb1b8bf1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:15.475753 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2475161c-cb5a-4e69-b143-59ca39a0d778 req-0c5727b0-ac5a-4916-bd27-af00eb1b8bf1 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 1096 time: 0.0928385 May 31 11:59:15.649773 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2475161c-cb5a-4e69-b143-59ca39a0d778 req-05cfc956-637b-4212-9747-046fbe989274 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:15.651554 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2475161c-cb5a-4e69-b143-59ca39a0d778 req-05cfc956-637b-4212-9747-046fbe989274 service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&shared=False HTTP/1.1" status: 200 len: 917 time: 0.1667635 May 31 11:59:15.849213 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2475161c-cb5a-4e69-b143-59ca39a0d778 req-2dc61519-c0e2-4723-8c2c-0af61fd02ede service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:15.850678 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2475161c-cb5a-4e69-b143-59ca39a0d778 req-2dc61519-c0e2-4723-8c2c-0af61fd02ede service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1924922 May 31 11:59:15.858786 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e5a0007-6dad-42f8-9c59-70092d934bde req-fc18095d-a872-47a8-9919-1414bb553244 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:15.860793 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1e5a0007-6dad-42f8-9c59-70092d934bde req-fc18095d-a872-47a8-9919-1414bb553244 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.1811271 May 31 11:59:15.933477 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ebda0b6-1035-4b04-89ed-00f807568d3a req-daf98263-2929-4034-a774-e0216002e0ef tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:15.936662 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4ebda0b6-1035-4b04-89ed-00f807568d3a req-daf98263-2929-4034-a774-e0216002e0ef tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.2738543 May 31 11:59:15.951286 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1e5a0007-6dad-42f8-9c59-70092d934bde req-c957acba-51dd-4462-825b-40b5802c0855 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0842068 May 31 11:59:15.984500 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4ebda0b6-1035-4b04-89ed-00f807568d3a req-a6f3dfb8-0eb5-4e4b-96a6-0e9a9b079ee0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0431807 May 31 11:59:15.988601 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1c6d376a-a0a7-4554-8d63-24c3a7f023a4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:15.990325 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1c6d376a-a0a7-4554-8d63-24c3a7f023a4 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 1096 time: 0.0927360 May 31 11:59:16.060481 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-1afc2954-068c-4571-b1b6-81f3829d5b9e service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=24163e07-db8d-4a12-8f62-285a1abb77fa HTTP/1.1" status: 200 len: 217 time: 0.0620322 May 31 11:59:16.128549 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-7d09646c-dc69-41e0-bd33-7d60f95633a1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:16.130118 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-7d09646c-dc69-41e0-bd33-7d60f95633a1 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0615406 May 31 11:59:16.209306 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-e78ed564-e465-40f5-a684-2f8da74c92f3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:16.210840 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-e78ed564-e465-40f5-a684-2f8da74c92f3 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0739441 May 31 11:59:16.425042 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-3807d67e-230b-4224-9e47-0e6b3197c550 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2063460 May 31 11:59:16.559605 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-97efd81d-e6d1-4c35-9868-8b60fd56bb9f req-94f34ef9-813f-4754-a718-b7c17695043d service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1281025 May 31 11:59:17.164355 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2ef4d51-15f9-4b8c-b5a4-ef19971f4a6c req-dc244b53-39b6-4d46-97c0-94740f527271 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:17.165475 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e2ef4d51-15f9-4b8c-b5a4-ef19971f4a6c req-dc244b53-39b6-4d46-97c0-94740f527271 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.1006029 May 31 11:59:17.189696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c8abfe5-8651-43f6-8aa5-6f47540a4d25 req-1da37d55-d74e-4b45-87d6-f60316adc48b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:17.191197 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9c8abfe5-8651-43f6-8aa5-6f47540a4d25 req-1da37d55-d74e-4b45-87d6-f60316adc48b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0905979 May 31 11:59:17.215936 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e2ef4d51-15f9-4b8c-b5a4-ef19971f4a6c req-6baeade7-70ae-44fb-aa0a-e83a7cbc6bc3 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0436594 May 31 11:59:17.237588 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9c8abfe5-8651-43f6-8aa5-6f47540a4d25 req-17ccf436-db63-4d73-bc83-831b0cd04196 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=4555883c-aba9-4fc6-894a-6ddaa27e8399&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0405505 May 31 11:59:17.543326 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-e4720892-cc0b-4031-bb31-6bfa41a78b84 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:17.544740 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-e4720892-cc0b-4031-bb31-6bfa41a78b84 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 945 time: 0.0831995 May 31 11:59:17.685002 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Deleting port d447eadc-67c7-4eca-8295-e5ae670ce616 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:17.685619 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:17.826347 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] For port d447eadc-67c7-4eca-8295-e5ae670ce616, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=d447eadc-67c7-4eca-8295-e5ae670ce616,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:17.826817 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:18.124285 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:18.124647 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:18.124959 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Floating IP 271403e9-87b5-4d80-b8da-95aafb8d1875 disassociated. External IP: 172.27.1.102, port: d447eadc-67c7-4eca-8295-e5ae670ce616. May 31 11:59:18.125294 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Calling delete_port for d447eadc-67c7-4eca-8295-e5ae670ce616 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:18.325048 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:18.331472 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:18.368090 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d644113-b30f-46f8-9e19-03f59e7d259a req-fe53d10d-f685-4cc9-9bf7-5c273556c316 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:18.369553 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6d644113-b30f-46f8-9e19-03f59e7d259a req-fe53d10d-f685-4cc9-9bf7-5c273556c316 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.0885870 May 31 11:59:18.410107 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event deleted for resources: {'Port': ['ID=d447eadc-67c7-4eca-8295-e5ae670ce616,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:18.416093 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6d644113-b30f-46f8-9e19-03f59e7d259a req-237a4d66-5084-4d12-bd24-9794f2f2013b tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404248 May 31 11:59:18.418035 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:18.582751 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a227a290-d61f-4ea0-8735-bd2efd9dd2e0 req-eb9e5ea2-7e87-400b-b9de-8f325663157a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 210 time: 0.0584843 May 31 11:59:18.621114 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5994148-ef51-4b4f-86b0-9f82244031e0 req-25a3ef57-4d26-4afd-b420-998ef12c6aac tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:18.622583 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d5994148-ef51-4b4f-86b0-9f82244031e0 req-25a3ef57-4d26-4afd-b420-998ef12c6aac tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.2550912 May 31 11:59:18.675022 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Router 6c5aae18-4faa-4947-af35-4078d428c2b9 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:59:18.681965 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d5994148-ef51-4b4f-86b0-9f82244031e0 req-d26fba79-ecd3-4c95-be57-bc6e9ade2e43 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0534217 May 31 11:59:18.888722 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:59:18.973543 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:18.989786 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] Sync routers for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:19.017805 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:19.018562 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:58:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:19.053809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] No DVR routers for this DVR port d447eadc-67c7-4eca-8295-e5ae670ce616 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:59:19.056528 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:19.059336 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'name': 'network-vif-deleted', 'tag': 'd447eadc-67c7-4eca-8295-e5ae670ce616'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:19.061646 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-e67e5a86-9683-4549-8c47-35121227871a" -d '{"events": [{"server_uuid": "f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6", "name": "network-vif-deleted", "tag": "d447eadc-67c7-4eca-8295-e5ae670ce616"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:19.119439 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:59:19 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-73d11d2f-233c-4779-aef8-f53af91b7db5 x-openstack-request-id: req-73d11d2f-233c-4779-aef8-f53af91b7db5 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:19.119812 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6", "name": "network-vif-deleted", "tag": "d447eadc-67c7-4eca-8295-e5ae670ce616", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:19.120100 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-73d11d2f-233c-4779-aef8-f53af91b7db5 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:19.120525 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6', 'name': 'network-vif-deleted', 'tag': 'd447eadc-67c7-4eca-8295-e5ae670ce616', 'status': 'completed', 'code': 200} May 31 11:59:19.121157 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:19.137781 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-54e477b1-e247-4515-be8d-d3fd0457c903 req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "DELETE /v2.0/ports/d447eadc-67c7-4eca-8295-e5ae670ce616 HTTP/1.1" status: 204 len: 173 time: 1.5872834 May 31 11:59:19.141184 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'port_id': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'mac_address': 'fa:16:3e:6f:6d:78', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 371, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:59:19.288303 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:19.308621 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:19.358369 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 24163e07-db8d-4a12-8f62-285a1abb77fa {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:59:19.481493 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:19.666626 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:19.671360 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:19.755983 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-baf24738-0e0d-48fc-9a93-17c0f1e4ffed req-5b7fd31e-6572-41b5-9829-3786aa3e7a93 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:19.759788 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-baf24738-0e0d-48fc-9a93-17c0f1e4ffed req-5b7fd31e-6572-41b5-9829-3786aa3e7a93 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Synchronizing usage tracker for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:19.780144 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:19.789316 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.789769 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.790175 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.790578 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.791038 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.791433 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.791881 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.792333 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.792738 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.793111 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.793483 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.800094 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:19.800094 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:19.812854 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:19.817192 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] Checking router: 6c5aae18-4faa-4947-af35-4078d428c2b9 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:19.830411 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:19.835045 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:58:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:19.856519 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-baf24738-0e0d-48fc-9a93-17c0f1e4ffed req-5b7fd31e-6572-41b5-9829-3786aa3e7a93 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Unset dirty status for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:19.857742 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-baf24738-0e0d-48fc-9a93-17c0f1e4ffed req-5b7fd31e-6572-41b5-9829-3786aa3e7a93 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.3082550 May 31 11:59:19.863209 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7b37af59-d1fb-41f1-bbbb-ba0470340dd9 req-cb79e117-7018-433f-b5f2-8e6630eccdba tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=f2c1bd8d-34cd-4a51-9f33-fcd684a7bff6 HTTP/1.1" status: 200 len: 210 time: 0.0532644 May 31 11:59:19.955939 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-baf24738-0e0d-48fc-9a93-17c0f1e4ffed req-3139b8a1-b425-4553-ab50-a3e2e0c1c403 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Synchronizing usage tracker for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:20.009947 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e195b117-e1dc-4155-a90a-b67fda219894 req-d7bbe1b5-7e61-4763-90e3-2edb834adb06 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:20.010945 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e195b117-e1dc-4155-a90a-b67fda219894 req-d7bbe1b5-7e61-4763-90e3-2edb834adb06 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.47 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 945 time: 0.1102204 May 31 11:59:20.021041 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['24163e07-db8d-4a12-8f62-285a1abb77fa'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:59:20.127540 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:20.180306 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Deleting port 3c46c8f8-777e-4ff2-8577-976c595eec9e {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:20.181237 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:20.193708 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-21d00929-ba98-474f-8588-cd8e6f1c6a00 req-6dc2996d-5252-42ef-8702-a3a9bd3d9c14 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:20.195690 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-21d00929-ba98-474f-8588-cd8e6f1c6a00 req-6dc2996d-5252-42ef-8702-a3a9bd3d9c14 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 944 time: 0.2067535 May 31 11:59:20.252734 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-21d00929-ba98-474f-8588-cd8e6f1c6a00 req-1d9f8f1f-c43c-4518-91be-fe3340b0b147 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Synchronizing usage tracker for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:security_group {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:20.261715 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-baf24738-0e0d-48fc-9a93-17c0f1e4ffed req-3139b8a1-b425-4553-ab50-a3e2e0c1c403 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Unset dirty status for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:20.266766 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-baf24738-0e0d-48fc-9a93-17c0f1e4ffed req-3139b8a1-b425-4553-ab50-a3e2e0c1c403 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/security-groups?id=7392633c-3cf7-4366-bb3e-fd78503c0a30&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.4022698 May 31 11:59:20.351669 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] For port 3c46c8f8-777e-4ff2-8577-976c595eec9e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3c46c8f8-777e-4ff2-8577-976c595eec9e,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:20.352189 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:20.455314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-21d00929-ba98-474f-8588-cd8e6f1c6a00 req-1d9f8f1f-c43c-4518-91be-fe3340b0b147 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Unset dirty status for tenant:0e7f0a333ee54b31bcd5c2290c3e284d on resource:security_group {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:20.456661 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-21d00929-ba98-474f-8588-cd8e6f1c6a00 req-1d9f8f1f-c43c-4518-91be-fe3340b0b147 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2548089 May 31 11:59:20.480941 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Calling delete_port for 3c46c8f8-777e-4ff2-8577-976c595eec9e owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:20.550750 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:20.554848 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-af4eb097-f19a-476b-b516-796a18d92ca2 None None] The sync data for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi is: [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:20.720240 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:20.730240 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:20.749100 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['3c46c8f8-777e-4ff2-8577-976c595eec9e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:59:20.784748 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ced0c33a-6f31-4a9b-b975-5e45896cac68 None None] New status for floating IP 271403e9-87b5-4d80-b8da-95aafb8d1875: DOWN {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 11:59:20.837006 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Pushing event deleted for resources: {'Port': ['ID=3c46c8f8-777e-4ff2-8577-976c595eec9e,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:20.878660 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.879019 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.882434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:20.890076 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.160s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:20.891508 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] No DVR routers for this DVR port 3c46c8f8-777e-4ff2-8577-976c595eec9e on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:59:20.893988 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:20.894913 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:20.945290 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-4d2e33d2-4dd1-4288-aa75-3e989dee529d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.004045 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e195b117-e1dc-4155-a90a-b67fda219894 req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.47 "DELETE /v2.0/ports/3c46c8f8-777e-4ff2-8577-976c595eec9e HTTP/1.1" status: 204 len: 173 time: 0.9869432 May 31 11:59:21.084864 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 24163e07-db8d-4a12-8f62-285a1abb77fa completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:21.090374 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 24163e07-db8d-4a12-8f62-285a1abb77fa triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:21.090486 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.096287 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36230) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:21.121919 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'name': 'network-vif-deleted', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:21.124136 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-dfdec6f3-755b-4758-8141-d782bde0e8c3" -d '{"events": [{"server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "name": "network-vif-deleted", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:21.167559 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3480d48a-a71a-4b7f-93f4-fe29ba8ee4c2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.180020 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:59:21 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-95fbab62-f80c-4e52-bfe2-a2a2501a60b8 x-openstack-request-id: req-95fbab62-f80c-4e52-bfe2-a2a2501a60b8 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:21.180340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "175ced20-c6a2-4bec-82db-394e58bd641e", "name": "network-vif-deleted", "tag": "3c46c8f8-777e-4ff2-8577-976c595eec9e", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:21.180610 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-95fbab62-f80c-4e52-bfe2-a2a2501a60b8 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:21.181017 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '175ced20-c6a2-4bec-82db-394e58bd641e', 'name': 'network-vif-deleted', 'tag': '3c46c8f8-777e-4ff2-8577-976c595eec9e', 'status': 'completed', 'code': 200} May 31 11:59:21.211872 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.216561 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3480d48a-a71a-4b7f-93f4-fe29ba8ee4c2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.240232 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:21.242542 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-da329bde-0206-4bde-a34e-996115f99195" -d '{"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-plugged", "status": "completed", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:21.250716 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:21.316314 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:59:21 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-5f8d9616-a497-4981-8923-691687e12962 x-openstack-request-id: req-5f8d9616-a497-4981-8923-691687e12962 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:21.316688 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-plugged", "status": "completed", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:21.316949 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-5f8d9616-a497-4981-8923-691687e12962 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:21.317351 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'code': 200} May 31 11:59:21.387753 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.388753 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device d447eadc-67c7-4eca-8295-e5ae670ce616 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:21.418201 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:21.423272 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3480d48a-a71a-4b7f-93f4-fe29ba8ee4c2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.432452 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:21.443803 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-25855d52-4f6f-4186-8bd7-fce1efc77414 req-8b6e4925-22eb-4d58-986f-d25a12dbd571 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Synchronizing usage tracker for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:21.452600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3480d48a-a71a-4b7f-93f4-fe29ba8ee4c2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=4555883c-aba9-4fc6-894a-6ddaa27e8399,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:21.456201 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: d447eadc-67c7-4eca-8295-e5ae670ce616 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:21.456746 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device d447eadc-67c7-4eca-8295-e5ae670ce616 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:21.457035 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3480d48a-a71a-4b7f-93f4-fe29ba8ee4c2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:21.507158 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:21.510149 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:21.594954 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-25855d52-4f6f-4186-8bd7-fce1efc77414 req-8b6e4925-22eb-4d58-986f-d25a12dbd571 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Unset dirty status for tenant:39ae37364a184ae7ae341ea7b2847c35 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:21.605065 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-25855d52-4f6f-4186-8bd7-fce1efc77414 req-8b6e4925-22eb-4d58-986f-d25a12dbd571 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] 10.222.0.48 "GET /v2.0/ports?device_id=175ced20-c6a2-4bec-82db-394e58bd641e HTTP/1.1" status: 200 len: 210 time: 0.2199626 May 31 11:59:21.621415 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-3480d48a-a71a-4b7f-93f4-fe29ba8ee4c2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.625702 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-3480d48a-a71a-4b7f-93f4-fe29ba8ee4c2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "DELETE /v2.0/security-groups/4555883c-aba9-4fc6-894a-6ddaa27e8399 HTTP/1.1" status: 204 len: 168 time: 0.5263703 May 31 11:59:21.626459 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36242) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:21.644192 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.api.v2.resource [None req-f13715f5-749c-4622-8b04-c1303ce4cd04 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] show failed (client error): The resource could not be found. May 31 11:59:21.645312 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-f13715f5-749c-4622-8b04-c1303ce4cd04 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups/4555883c-aba9-4fc6-894a-6ddaa27e8399 HTTP/1.1" status: 404 len: 347 time: 0.0176508 May 31 11:59:21.648260 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36246) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:21.697109 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4421cd21-b1d6-496c-86c9-1f4c9d3dae23 req-94595e8d-84f3-4649-a822-3041d6f55cc2 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:21.698624 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4421cd21-b1d6-496c-86c9-1f4c9d3dae23 req-94595e8d-84f3-4649-a822-3041d6f55cc2 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.1112046 May 31 11:59:21.746290 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4421cd21-b1d6-496c-86c9-1f4c9d3dae23 req-595878eb-7087-4547-96e7-003e7c15f113 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414000 May 31 11:59:21.935691 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Deleting port 4f8bed93-6c8d-4aaf-ac82-20f075c5d327 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:21.936038 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:21.952289 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d829272-51b7-4a04-9b4d-f9e9fc6b2181 req-45f9239a-e001-42b6-8355-d9e1c818ef8a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:21.953851 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7d829272-51b7-4a04-9b4d-f9e9fc6b2181 req-45f9239a-e001-42b6-8355-d9e1c818ef8a service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 1098 time: 0.1107690 May 31 11:59:22.166294 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7d829272-51b7-4a04-9b4d-f9e9fc6b2181 req-0e4b4eac-a339-4b7d-958d-527c19917c22 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0683091 May 31 11:59:22.186288 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:22.186608 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:22.310746 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Calling delete_port for 4f8bed93-6c8d-4aaf-ac82-20f075c5d327 owned by network:floatingip {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:22.502480 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:22.507340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:22.554400 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event deleted for resources: {'Port': ['ID=4f8bed93-6c8d-4aaf-ac82-20f075c5d327,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:22.557487 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:22.647805 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:22.662465 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:22.663676 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "DELETE /v2.0/floatingips/271403e9-87b5-4d80-b8da-95aafb8d1875 HTTP/1.1" status: 204 len: 168 time: 1.0143831 May 31 11:59:22.899693 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 3c46c8f8-777e-4ff2-8577-976c595eec9e no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:22.994135 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 3c46c8f8-777e-4ff2-8577-976c595eec9e {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:22.994667 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 3c46c8f8-777e-4ff2-8577-976c595eec9e not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:23.041382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a28c2cac-d1d2-467a-949b-2a137aa78b4a req-1259d9b9-a300-40ec-815d-60f3b609ac2a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:23.042863 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a28c2cac-d1d2-467a-949b-2a137aa78b4a req-1259d9b9-a300-40ec-815d-60f3b609ac2a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.1150703 May 31 11:59:23.087402 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a28c2cac-d1d2-467a-949b-2a137aa78b4a req-59a393d8-27c5-478d-943a-5e378bfc0f7b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0378931 May 31 11:59:23.365225 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:23.375686 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36298) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:23.381981 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'security_group': {'name': 'tempest-securitygroup--539867399', 'description': 'tempest-description--664445151'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:23.387880 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:23.393246 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:security_group and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:23.397523 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:23.405739 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:23.513375 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-7d5c0137-c1de-45f2-b05d-be143057ff25 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:23.542071 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:23.563841 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:23.567553 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:security_group_rule and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:23.570536 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:23.577312 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:23.651063 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:23.730342 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:23.736296 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8684b399-1939-4742-98c0-bbef648c5499 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:23.775945 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'SecurityGroup': ['ID=237a0cf4-5be9-4442-a0d4-170bc2d699a7,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:23.778662 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:23.879098 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:23.880262 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1778 time: 0.5026798 May 31 11:59:23.883083 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36324) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:23.888166 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'security_group_rule': {'security_group_id': '237a0cf4-5be9-4442-a0d4-170bc2d699a7', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:23.894855 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:23.898684 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:security_group_rule and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:23.901518 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:23.907450 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:23.991221 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.055982 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.189017 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.191206 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:24.218654 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=85d3dd34-d6a2-468c-8600-e4362962de42,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:24.223793 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:24.280568 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e8b51a14-1f3b-4458-8724-e69a7da14e81 req-3ed4e6ac-9405-4758-bc95-16b5e97b9ffa tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:24.282605 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e8b51a14-1f3b-4458-8724-e69a7da14e81 req-3ed4e6ac-9405-4758-bc95-16b5e97b9ffa tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0841997 May 31 11:59:24.325522 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e8b51a14-1f3b-4458-8724-e69a7da14e81 req-32bec7fe-6b32-4dd2-b65b-bbd27b61e607 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0370111 May 31 11:59:24.380482 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.384049 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d76f3104-676c-4f13-aec5-d7a8e615bb67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.4975383 May 31 11:59:24.384524 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36340) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:24.389764 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'security_group_rule': {'security_group_id': '237a0cf4-5be9-4442-a0d4-170bc2d699a7', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:24.396763 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:24.401471 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:security_group_rule and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:24.404563 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:24.410925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:24.513993 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ded6965-6148-4e9d-8de5-a630a6cf7275 req-3c1667dc-ddc9-45f7-80fd-fd65da44f61b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:24.515538 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0ded6965-6148-4e9d-8de5-a630a6cf7275 req-3c1667dc-ddc9-45f7-80fd-fd65da44f61b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0843425 May 31 11:59:24.529652 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36344) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:24.534749 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Request body: {'floatingip': {'port_id': '24163e07-db8d-4a12-8f62-285a1abb77fa'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:24.588311 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.616554 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.621213 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.656457 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.741908 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.749742 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e2947a57-e86d-467b-8438-85bed8aa6054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:24.770701 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=cf60b29a-9707-4b45-a89d-25d6bcaddbc8,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:24.773105 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:24.830608 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36350) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:24.972663 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:24.973753 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.5881534 May 31 11:59:24.976888 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36354) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:24.983916 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:24.993679 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:25.002631 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Usage tracker for resource:floatingip and tenant:b866e6eab1b04406a7280f91698983ce is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:25.006993 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:25.078479 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:25.395225 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:25.418235 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:25.457810 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:25.609762 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:25.824300 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.l3_db [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Floating IP e6d08467-4c4b-4519-a294-dd4711658a52 associated. External IP: 172.27.1.9, port: 24163e07-db8d-4a12-8f62-285a1abb77fa. May 31 11:59:25.916588 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:25.923666 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Deleting port 6b60438b-78d8-4039-a0ce-c42d04559f08 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:25.924004 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.046345 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] For port 6b60438b-78d8-4039-a0ce-c42d04559f08, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=6b60438b-78d8-4039-a0ce-c42d04559f08,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:26.046697 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.113805 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.170045 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Calling delete_port for 6b60438b-78d8-4039-a0ce-c42d04559f08 owned by network:router_interface {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:26.312443 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Allocated IP 172.27.1.248 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/199cf35d-e696-43da-9032-c8ab40b0707d) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:26.422092 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.427466 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3222c8dc-5b3d-4b05-b9e8-f4945b03855b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:26.429733 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:59:26.463886 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36370) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:26.519023 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Pushing event deleted for resources: {'Port': ['ID=6b60438b-78d8-4039-a0ce-c42d04559f08,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:26.531355 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:26.682891 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:26.692636 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] Sync routers for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:26.702413 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.712269 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:26.712269 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:26.724883 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.744152 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.748921 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:26.775453 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d56930f8-4f11-4009-9524-9d3b7710a922 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "PUT /v2.0/floatingips/e6d08467-4c4b-4519-a294-dd4711658a52 HTTP/1.1" status: 200 len: 989 time: 2.2420270 May 31 11:59:26.775453 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:26.776924 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783" -d '{"events": [{"name": "network-changed", "server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:26.801906 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Usage tracker for resource:security_group_rule and tenant:b8428fe361ef41a785c0db5976a6e724 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:26.812321 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:26.832283 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:26.849788 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:59:26 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-33350d27-0fc9-4976-af95-518b788e9373 x-openstack-request-id: req-33350d27-0fc9-4976-af95-518b788e9373 {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:26.850109 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:26.850413 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-33350d27-0fc9-4976-af95-518b788e9373 {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:26.850855 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'status': 'completed', 'code': 200} May 31 11:59:26.869895 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.891799 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:26.900231 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-26048555-c9f9-46ef-b931-51d6c1753098 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:26.957385 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:26.991570 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:26.994593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:26.998715 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:27.036388 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'Port': ['ID=199cf35d-e696-43da-9032-c8ab40b0707d,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:27.040933 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.143s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:27.088971 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:27.186051 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:27.247834 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-575c87a1-4e0a-4b58-b93d-ab750d7a3993 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:27.249306 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-575c87a1-4e0a-4b58-b93d-ab750d7a3993 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 1098 time: 0.2943392 May 31 11:59:27.259071 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:27.359320 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'e6d08467-4c4b-4519-a294-dd4711658a52', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'floating_ip_address': '172.27.1.9', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'port_id': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 433, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'mac_address': 'fa:16:3e:6f:6d:78', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:59:07Z', 'updated_at': '2021-05-31T11:59:25Z', 'revision_number': 1, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:27.364770 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] Checking router: 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:27.374941 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:27.376578 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:27.531342 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:27.539983 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Router d195a538-22a1-483f-a3e5-9007c6338ece has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:59:27.543806 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b65cfadb-5201-448c-9dad-b8d5f64c8ca9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:27.613414 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:27.617174 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:27.707960 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=554af37c-10ac-4c2f-84aa-7d6ee6c46eea,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:27.713285 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:27.722684 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-47f1514e-31cb-482e-9778-5cf381c269d8 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:27.725407 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=b8428fe361ef41a785c0db5976a6e724&name=default HTTP/1.1" status: 200 len: 2979 time: 1.2588246 May 31 11:59:27.726453 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.183s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:27.741260 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36388) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:27.849309 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'Port': ['ID=199cf35d-e696-43da-9032-c8ab40b0707d,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:27.854945 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:27.856182 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:27.878493 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-1bb38e35-fc51-4da3-b6b5-07c764e1c8b3 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:27.879939 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-1bb38e35-fc51-4da3-b6b5-07c764e1c8b3 service neutron] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 917 time: 0.6198974 May 31 11:59:27.976172 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:27.978293 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 3.0004699 May 31 11:59:28.024169 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-afb9f8ee-2263-4281-993b-ca0a34866a3a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:28.026169 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:28.038259 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] Sync routers for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:28.043825 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-74f9907a-5f38-40d8-ba43-373faffb543e service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=24163e07-db8d-4a12-8f62-285a1abb77fa HTTP/1.1" status: 200 len: 997 time: 0.1545064 May 31 11:59:28.057589 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:28.060124 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:28.066274 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-afb9f8ee-2263-4281-993b-ca0a34866a3a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:28.075223 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'e6d08467-4c4b-4519-a294-dd4711658a52', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'floating_ip_address': '172.27.1.9', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'port_id': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 433, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'mac_address': 'fa:16:3e:6f:6d:78', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:59:07Z', 'updated_at': '2021-05-31T11:59:25Z', 'revision_number': 1, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:28.080967 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d9c7bd89-ed4f-4eb0-8385-6dfc33fe81d6 None None] The sync data for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi is: [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'e6d08467-4c4b-4519-a294-dd4711658a52', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'floating_ip_address': '172.27.1.9', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'port_id': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 433, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'mac_address': 'fa:16:3e:6f:6d:78', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:59:07Z', 'updated_at': '2021-05-31T11:59:25Z', 'revision_number': 1, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:28.183069 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-af636651-a4cf-49d0-80df-ae7f39d26e2b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:28.184523 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-af636651-a4cf-49d0-80df-ae7f39d26e2b service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.1114964 May 31 11:59:28.236457 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] 10.222.0.48 "PUT /v2.0/routers/d195a538-22a1-483f-a3e5-9007c6338ece/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.4008489 May 31 11:59:28.236880 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36406) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:28.300749 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-a6f7710d-b7a4-43ce-85e4-ef1bea002fc0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:28.302245 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-a6f7710d-b7a4-43ce-85e4-ef1bea002fc0 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1086934 May 31 11:59:28.330875 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-2736ae36-c669-415c-8d46-ed29b44f6e1b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:28.340654 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-afb9f8ee-2263-4281-993b-ca0a34866a3a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:28.356838 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8db5fa03-5b87-44cc-ab40-3de69a0a721a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:28.357031 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:28.430110 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-eec7af89-c59f-4b72-baaa-fafdb33789a2 None None] New status for floating IP e6d08467-4c4b-4519-a294-dd4711658a52: ACTIVE {{(pid=99041) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 11:59:28.445213 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-afb9f8ee-2263-4281-993b-ca0a34866a3a admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:28.446592 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-afb9f8ee-2263-4281-993b-ca0a34866a3a admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/554af37c-10ac-4c2f-84aa-7d6ee6c46eea HTTP/1.1" status: 204 len: 168 time: 0.7043626 May 31 11:59:28.450020 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afb9f8ee-2263-4281-993b-ca0a34866a3a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=554af37c-10ac-4c2f-84aa-7d6ee6c46eea,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:28.465404 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-afb9f8ee-2263-4281-993b-ca0a34866a3a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:28.502737 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks [] for router, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:28.513965 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-60b864af-f0ab-404e-af97-6654ea9c52aa service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2030737 May 31 11:59:28.543849 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-2736ae36-c669-415c-8d46-ed29b44f6e1b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:28.545274 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-2736ae36-c669-415c-8d46-ed29b44f6e1b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-securitygroup--539867399&fields=id&tenant_id=b866e6eab1b04406a7280f91698983ce HTTP/1.1" status: 200 len: 267 time: 0.2803798 May 31 11:59:28.722035 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:28.763540 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214'}], 'enable_snat': True}, 'gw_port_id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:59:26Z', 'revision_number': 5, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port': {'id': '1b4b9cf5-650e-45b0-b762-4b42ac55ad1d', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:53:80:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'device_owner': 'network:router_gateway', 'standard_attr_id': 208, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.214', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:00Z', 'updated_at': '2021-05-31T11:56:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:28.768507 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] Checking router: d195a538-22a1-483f-a3e5-9007c6338ece for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:28.778023 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] Routers {'d195a538-22a1-483f-a3e5-9007c6338ece'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:28.778727 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['d195a538-22a1-483f-a3e5-9007c6338ece'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:28.897703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Deleting port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:28.898317 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:28.903629 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f849c3f-7fe5-4dfc-b29f-fb444afb6783 req-92bccfd4-d673-4cf0-af50-e506ce985cc2 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3838651 May 31 11:59:28.949925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-06dcc46d-9063-41ed-a511-1189381eb708 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:28.950654 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-06dcc46d-9063-41ed-a511-1189381eb708 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:network {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:28.962421 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] Set router d195a538-22a1-483f-a3e5-9007c6338ece gateway port host: None {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:29.085478 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:59:28Z', 'revision_number': 6, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port_host': None}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:29.089832 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b33f68be-045a-4d04-b79d-e5f2e9f42226 None None] The sync data for ids ['d195a538-22a1-483f-a3e5-9007c6338ece'] in dv-cinder-791239-4-iscsi is: [{'id': 'd195a538-22a1-483f-a3e5-9007c6338ece', 'name': 'tempest-TestMinimumBasicScenario-404047115-router', 'tenant_id': '39ae37364a184ae7ae341ea7b2847c35', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:55:58Z', 'updated_at': '2021-05-31T11:59:28Z', 'revision_number': 6, 'project_id': '39ae37364a184ae7ae341ea7b2847c35', 'gw_port_host': None}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:29.102098 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] For port 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=1b4b9cf5-650e-45b0-b762-4b42ac55ad1d,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:29.102452 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:29.157354 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-06dcc46d-9063-41ed-a511-1189381eb708 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:network {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:29.158595 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-06dcc46d-9063-41ed-a511-1189381eb708 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 811 time: 0.6052220 May 31 11:59:29.176824 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-048d615a-166c-4063-9071-a6d06aa164fb tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/quotas/b866e6eab1b04406a7280f91698983ce HTTP/1.1" status: 200 len: 375 time: 0.0113182 May 31 11:59:29.231041 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Calling delete_port for 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d owned by network:router_gateway {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:29.267404 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a4a1edc7-5f8b-437d-a772-179182c674b3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:29.394943 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a4a1edc7-5f8b-437d-a772-179182c674b3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:29.395299 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a4a1edc7-5f8b-437d-a772-179182c674b3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&fields=id HTTP/1.1" status: 200 len: 348 time: 0.2130306 May 31 11:59:29.456590 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:29.461762 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:29.508425 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36426) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:29.542778 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Pushing event deleted for resources: {'Port': ['ID=1b4b9cf5-650e-45b0-b762-4b42ac55ad1d,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:29.545492 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:29.557937 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:29.624456 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:29.640280 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Usage tracker for resource:security_group_rule and tenant:1c7608ac101f40378b9d1da51610a200 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:29.645419 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:29.648147 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:29.653776 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:29.813953 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-94179d0c-8cdc-4e4c-be1e-180b3434183b service neutron] 10.222.0.48 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4071629 May 31 11:59:29.933986 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:29.936933 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:29.996735 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-fe22d246-b78a-4e34-9fd2-cd59741808d8 service neutron] 10.222.0.48 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1758587 May 31 11:59:29.998492 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.044751 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.049466 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Fanout notify agent at l3_agent the message router_deleted on router d195a538-22a1-483f-a3e5-9007c6338ece {{(pid=99039) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 11:59:30.057618 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.061285 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] 10.222.0.48 "DELETE /v2.0/routers/d195a538-22a1-483f-a3e5-9007c6338ece HTTP/1.1" status: 204 len: 168 time: 1.8208842 May 31 11:59:30.061622 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36430) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:30.097943 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.108868 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-dc687b6a-742c-4754-b117-c223757d83b0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:30.173917 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_v2 [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Deleting subnet 672ec900-b55c-44a0-9ed9-7e27beeace12 {{(pid=99040) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 11:59:30.183453 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=1c7608ac101f40378b9d1da51610a200&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6692317 May 31 11:59:30.183453 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0249da70-9558-4267-8d95-52bf1c8306d3,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:30.183453 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.184690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36436) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:30.211008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:30.265978 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3096829'] for subnet, precommit_delete_associations {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.307690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-11401e52-664d-4e95-87ed-4eb912fd60a8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.337932 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-11401e52-664d-4e95-87ed-4eb912fd60a8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.384870 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d7ca0d4e-65db-4457-96de-f986be94b811 req-2ba5d3a7-b684-4d1d-89b5-1aef39dac86a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 210 time: 0.0750592 May 31 11:59:30.462149 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.462476 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-11401e52-664d-4e95-87ed-4eb912fd60a8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.470932 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-aa33ab48-d79e-46e8-bb88-89b75ce51c8f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:30.488409 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-11401e52-664d-4e95-87ed-4eb912fd60a8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0249da70-9558-4267-8d95-52bf1c8306d3,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:30.502768 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-11401e52-664d-4e95-87ed-4eb912fd60a8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:30.505462 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-11401e52-664d-4e95-87ed-4eb912fd60a8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.508807 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-11401e52-664d-4e95-87ed-4eb912fd60a8 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/0249da70-9558-4267-8d95-52bf1c8306d3 HTTP/1.1" status: 204 len: 168 time: 0.3237395 May 31 11:59:30.670803 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 6b60438b-78d8-4039-a0ce-c42d04559f08 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:30.762955 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 6b60438b-78d8-4039-a0ce-c42d04559f08 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:30.763301 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 6b60438b-78d8-4039-a0ce-c42d04559f08 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:30.763713 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:30.798576 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:30.799145 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 1b4b9cf5-650e-45b0-b762-4b42ac55ad1d not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:30.827688 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2111654', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2831123'] for subnet, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.833040 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2ac64b57-10a5-4407-b96d-43ff6a5ceea7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:30.867363 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=672ec900-b55c-44a0-9ed9-7e27beeace12,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:30.869999 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:30.871868 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:30.876641 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-384dd8d3-62d5-434f-9a1e-ca228a6457d0 None None] Network 0fae71a7-8e60-4f7d-a467-822e3d8d87fb requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:30.880838 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/672ec900-b55c-44a0-9ed9-7e27beeace12 HTTP/1.1" status: 204 len: 168 time: 0.8181543 May 31 11:59:30.883343 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36462) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:31.025267 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:31.121873 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Deleting port 447dd278-807b-43af-8f79-ec082f9f9049 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:31.122013 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:31.173642 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] DHCP port deletion for 0fae71a7-8e60-4f7d-a467-822e3d8d87fb request from dv-cinder-791239-4-iscsi {{(pid=99041) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 31 11:59:31.229257 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] For port 447dd278-807b-43af-8f79-ec082f9f9049, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=447dd278-807b-43af-8f79-ec082f9f9049,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:31.229658 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:31.243410 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] Deleting port 447dd278-807b-43af-8f79-ec082f9f9049 {{(pid=99041) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:31.243806 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:31.345328 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] For port 447dd278-807b-43af-8f79-ec082f9f9049, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=447dd278-807b-43af-8f79-ec082f9f9049,segment=NetworkSegment(061f4d79-99bf-47ba-a95e-2ecb79298c69),segment_id=061f4d79-99bf-47ba-a95e-2ecb79298c69)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:31.345887 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:31.377407 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Calling delete_port for 447dd278-807b-43af-8f79-ec082f9f9049 owned by network:dhcp {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:31.390261 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36470) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:31.423461 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:31.442638 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:31.448929 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Usage tracker for resource:security_group_rule and tenant:32d27ee9f0134a309e1854089d568636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:31.454380 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:31.464499 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:31.475114 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] Calling delete_port for 447dd278-807b-43af-8f79-ec082f9f9049 owned by network:dhcp {{(pid=99041) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:31.619651 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-53d0e352-9f24-434d-a926-83a6a2ccc23a req-68cba6cf-642b-4fd1-bd19-05595e9b1ab3 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 210 time: 0.0698080 May 31 11:59:31.628485 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:31.646095 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6f23d9e3-b2e5-45c0-8804-5987bfab2368 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:31.649454 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:31.653530 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f93ed1f6-564b-4dcb-8e56-42981f0e04ce None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:31.698217 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] Pushing event deleted for resources: {'Port': ['ID=447dd278-807b-43af-8f79-ec082f9f9049,revision_number=None']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:31.699056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] No DVR routers for this DVR port 447dd278-807b-43af-8f79-ec082f9f9049 on host dv-cinder-791239-4-iscsi {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:59:31.701690 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:31.704881 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] No DVR routers for this DVR port 447dd278-807b-43af-8f79-ec082f9f9049 on host dv-cinder-791239-4-iscsi {{(pid=99041) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:59:31.754535 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Pushing event deleted for resources: {'Port': ['ID=447dd278-807b-43af-8f79-ec082f9f9049,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:31.764523 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:31.988253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.000986 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.019862 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['0fae71a7-8e60-4f7d-a467-822e3d8d87fb']}} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:32.036254 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '061f4d79-99bf-47ba-a95e-2ecb79298c69') {'for_net_delete': True} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:32.036646 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '061f4d79-99bf-47ba-a95e-2ecb79298c69') {} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:32.051728 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3096822'] for segment, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.072158 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.078265 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Releasing vlan 366 on physical network physnet1 to pool {{(pid=99040) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 11:59:32.079204 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2278144'] for segment, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: Traceback (most recent call last): May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: timer() May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: cb(*args, **kw) May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: return func(*args, **kwargs) May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: self._notify() May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: self.callback(batched_events) May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: event.method(event) May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_uuid = self.p_client.list_aggregates( May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: return f(self, *a, **k) May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._get(url).json() May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: return self.request(url, 'GET', **kwargs) May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: base_url = self.get_endpoint(auth, allow=allow, May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 11:59:32.092010 dv-cinder-791239-4-iscsi neutron-server[99040]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 11:59:32.096415 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.108301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4e161ee1-b4fd-41d2-878b-dc8eaa57c48b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:32.206058 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=32d27ee9f0134a309e1854089d568636&name=default HTTP/1.1" status: 200 len: 2979 time: 0.8147440 May 31 11:59:32.209309 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36484) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:32.223701 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=34a3191e-9852-456e-8722-64b37628c21e,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:32.227790 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:32.279591 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5bb90993-094a-490a-aaa5-43c98a6b7b63 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.304340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.310489 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5bb90993-094a-490a-aaa5-43c98a6b7b63 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.325594 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-98755f07-fecc-4328-a34f-d7ad21c727d0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:32.344010 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.368043 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Pushing event deleted for resources: {'Network': ['ID=0fae71a7-8e60-4f7d-a467-822e3d8d87fb,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:32.375938 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:32.402071 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] 10.222.0.48 "DELETE /v2.0/networks/0fae71a7-8e60-4f7d-a467-822e3d8d87fb HTTP/1.1" status: 204 len: 168 time: 1.5175428 May 31 11:59:32.404919 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36488) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:32.530339 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5bb90993-094a-490a-aaa5-43c98a6b7b63 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.534857 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-4d18e40f-b503-4107-b314-0e83fe9db6de admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:32.545935 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5bb90993-094a-490a-aaa5-43c98a6b7b63 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=34a3191e-9852-456e-8722-64b37628c21e,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:32.549455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5bb90993-094a-490a-aaa5-43c98a6b7b63 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:32.555517 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5bb90993-094a-490a-aaa5-43c98a6b7b63 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:32.559401 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-5bb90993-094a-490a-aaa5-43c98a6b7b63 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/34a3191e-9852-456e-8722-64b37628c21e HTTP/1.1" status: 204 len: 168 time: 0.3465512 May 31 11:59:32.683842 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 447dd278-807b-43af-8f79-ec082f9f9049 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:32.738069 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 447dd278-807b-43af-8f79-ec082f9f9049 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:32.738699 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 447dd278-807b-43af-8f79-ec082f9f9049 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:32.870811 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-69ce0f3c-cfdb-486f-aa6c-82fd77292ef2 req-956619f6-d345-4aa6-b369-0094f9ac4f8f tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 210 time: 0.0772312 May 31 11:59:33.025754 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-3746ffcd-7129-4f7a-8e6f-42c3a66aea41 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:33.027506 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-3746ffcd-7129-4f7a-8e6f-42c3a66aea41 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 811 time: 0.1935823 May 31 11:59:33.090510 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-1c9d2708-c55f-4946-bec2-66f3420cd0fd tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=b866e6eab1b04406a7280f91698983ce HTTP/1.1" status: 200 len: 379 time: 0.0474522 May 31 11:59:33.108943 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'port': {'device_id': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'admin_state_up': True, 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'security_groups': ['237a0cf4-5be9-4442-a0d4-170bc2d699a7']}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:33.118975 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:33.255580 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:33.265089 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:33.372172 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Made reservation on behalf of b866e6eab1b04406a7280f91698983ce for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:33.381016 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:33.605835 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Deleting port 5c78fca6-f530-48c3-8190-890cb005d8f5 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:33.606471 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:33.612479 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Allocated IP 172.20.0.14 (36cce22a-502c-464d-a9ac-50f2bc247e07/1b74adba-cf2f-4db5-ad92-27ea39726767/6bd1051c-af78-4616-88bd-002db4d1b521) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:33.734386 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] For port 5c78fca6-f530-48c3-8190-890cb005d8f5, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=5c78fca6-f530-48c3-8190-890cb005d8f5,segment=NetworkSegment(0eba2510-55a0-4815-bcb1-2dc142aabf11),segment_id=0eba2510-55a0-4815-bcb1-2dc142aabf11)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:33.735404 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:33.883762 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:33.904560 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Transition to ACTIVE for port object 6bd1051c-af78-4616-88bd-002db4d1b521 will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:33.998305 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Calling delete_port for 5c78fca6-f530-48c3-8190-890cb005d8f5 owned by network:router_interface {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:34.061577 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:34.070017 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8f45872e-b09a-4e92-9a0a-5f96d6769fd4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:34.149336 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:34.153603 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:34.162290 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:34.192895 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event updated for resources: {'Port': ['ID=6bd1051c-af78-4616-88bd-002db4d1b521,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:34.207151 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:34.251406 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:34.252234 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:34.268699 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fee4c1b0-ee37-4b3d-ba79-0ca99e898a5f req-313911c6-9397-4dbd-8bb4-b0fc05b5afa2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:34.270252 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fee4c1b0-ee37-4b3d-ba79-0ca99e898a5f req-313911c6-9397-4dbd-8bb4-b0fc05b5afa2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 932 time: 0.2518015 May 31 11:59:34.333793 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fee4c1b0-ee37-4b3d-ba79-0ca99e898a5f req-80b675ce-c431-4e71-b77b-f7bf5c57e1aa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0545816 May 31 11:59:34.348001 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:34.361647 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:34.416617 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Pushing event deleted for resources: {'Port': ['ID=5c78fca6-f530-48c3-8190-890cb005d8f5,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:34.428234 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.3239973 May 31 11:59:34.441733 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Request body: {'port': {'device_id': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:34.482445 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.482982 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.483551 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.485998 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.486440 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.486916 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.487340 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.487745 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.488151 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.488554 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.489102 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.489604 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:34.553075 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-114df4d8-1067-4b51-b036-08e136f71914 None None] Provisioning for port 6bd1051c-af78-4616-88bd-002db4d1b521 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:34.560418 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-114df4d8-1067-4b51-b036-08e136f71914 None None] Provisioning complete for port 6bd1051c-af78-4616-88bd-002db4d1b521 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:34.563194 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-114df4d8-1067-4b51-b036-08e136f71914 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:34.573431 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.218s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:34.613584 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:34.757004 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-114df4d8-1067-4b51-b036-08e136f71914 None None] Port 6bd1051c-af78-4616-88bd-002db4d1b521 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:59:34.958690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:35.261390 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 6bd1051c-af78-4616-88bd-002db4d1b521 {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:59:35.269883 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:35.287156 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Transition to ACTIVE for port object 6bd1051c-af78-4616-88bd-002db4d1b521 will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:35.421471 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:35.426813 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3919621e-53c5-4662-a9a9-d2de34920fb1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:35.432717 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:59:35.502803 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 11:59:35.569366 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Pushing event updated for resources: {'Port': ['ID=6bd1051c-af78-4616-88bd-002db4d1b521,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:35.609493 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.183s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:35.718759 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:35.729788 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] Sync routers for ids ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:35.735198 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] 10.222.0.48 "PUT /v2.0/routers/4c61aefe-0df0-474e-8c1a-3fa776e16bcc/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.3264937 May 31 11:59:35.735523 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Attempting to bind port 6bd1051c-af78-4616-88bd-002db4d1b521 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:59:35.735523 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Attempting to bind port 6bd1051c-af78-4616-88bd-002db4d1b521 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:59:35.735523 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Attempting to bind port 6bd1051c-af78-4616-88bd-002db4d1b521 on network 36cce22a-502c-464d-a9ac-50f2bc247e07 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:59:35.737654 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36608) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:35.740996 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Port 6bd1051c-af78-4616-88bd-002db4d1b521 on network 36cce22a-502c-464d-a9ac-50f2bc247e07 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 11:59:35.741315 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Attempting to bind port 6bd1051c-af78-4616-88bd-002db4d1b521 on network 36cce22a-502c-464d-a9ac-50f2bc247e07 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:59:35.749229 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 59, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:59:35.751488 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Checking segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:35.751488 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Bound using segment: {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:59:35.751488 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Bound port: 6bd1051c-af78-4616-88bd-002db4d1b521, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 363, 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:59:35.751967 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:35.753684 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] Routers {'4c61aefe-0df0-474e-8c1a-3fa776e16bcc'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:35.754559 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:35.960468 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks [] for router, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:35.979877 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] For port 6bd1051c-af78-4616-88bd-002db4d1b521, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:59:35.989282 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] For port 6bd1051c-af78-4616-88bd-002db4d1b521, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6bd1051c-af78-4616-88bd-002db4d1b521,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:59:36.005729 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3f95c432-a5d2-459f-94a9-7262ad0bd429 req-8f36ec9a-df05-4b82-80d8-ec990cca77a6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:36.006809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3f95c432-a5d2-459f-94a9-7262ad0bd429 req-8f36ec9a-df05-4b82-80d8-ec990cca77a6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:36.024044 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] Set router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:36.107701 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3f95c432-a5d2-459f-94a9-7262ad0bd429 req-8f36ec9a-df05-4b82-80d8-ec990cca77a6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:36.118827 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3f95c432-a5d2-459f-94a9-7262ad0bd429 req-8f36ec9a-df05-4b82-80d8-ec990cca77a6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 944 time: 0.3456557 May 31 11:59:36.118827 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:36.118827 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:36.208662 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Pushing event updated for resources: {'Port': ['ID=6bd1051c-af78-4616-88bd-002db4d1b521,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:36.222842 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:36.285984 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:36.291881 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:36.302388 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] 10.222.0.47 "PUT /v2.0/ports/6bd1051c-af78-4616-88bd-002db4d1b521 HTTP/1.1" status: 200 len: 1094 time: 1.8579812 May 31 11:59:36.319527 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'tag': '6bd1051c-af78-4616-88bd-002db4d1b521'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:36.329491 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2" -d '{"events": [{"name": "network-changed", "server_uuid": "dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca", "tag": "6bd1051c-af78-4616-88bd-002db4d1b521"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:36.397348 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:36.420906 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3f95c432-a5d2-459f-94a9-7262ad0bd429 req-a43ec72d-dbcc-4f5f-aa09-53c6621c84d4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2153296 May 31 11:59:36.445542 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:59:36 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-d52b9f38-b831-48b5-a285-ab8789b8258c x-openstack-request-id: req-d52b9f38-b831-48b5-a285-ab8789b8258c {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:36.445542 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca", "tag": "6bd1051c-af78-4616-88bd-002db4d1b521", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:36.445542 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-d52b9f38-b831-48b5-a285-ab8789b8258c {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:36.447437 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'tag': '6bd1051c-af78-4616-88bd-002db4d1b521', 'status': 'completed', 'code': 200} May 31 11:59:36.499861 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:59:34Z', 'revision_number': 5, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:36.725966 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] Checking router: 4c61aefe-0df0-474e-8c1a-3fa776e16bcc for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:36.725966 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] Routers {'4c61aefe-0df0-474e-8c1a-3fa776e16bcc'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:36.725966 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:36.729172 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-e34e4660-101b-47dd-bdd7-34c111c5b561 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:36.729172 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-e34e4660-101b-47dd-bdd7-34c111c5b561 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 1097 time: 0.1074433 May 31 11:59:36.729172 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-5b163e59-2753-4ed7-81eb-17de7b62d82f service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=6bd1051c-af78-4616-88bd-002db4d1b521 HTTP/1.1" status: 200 len: 217 time: 0.1170363 May 31 11:59:36.743333 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Deleting port 4ccc4832-093a-44f0-a0bb-60f43b576309 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:36.751014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:36.770320 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-e8891f1a-7b2d-467c-87ef-285412fdd424 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:36.771789 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-e8891f1a-7b2d-467c-87ef-285412fdd424 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0682077 May 31 11:59:36.784972 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] Set router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:36.854042 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-c1414b71-6f26-4e69-8a1b-29622158ac0f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:36.855509 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-c1414b71-6f26-4e69-8a1b-29622158ac0f service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0749161 May 31 11:59:36.903186 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] For port 4ccc4832-093a-44f0-a0bb-60f43b576309, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=4ccc4832-093a-44f0-a0bb-60f43b576309,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:36.903538 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:37.061261 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Calling delete_port for 4ccc4832-093a-44f0-a0bb-60f43b576309 owned by network:router_gateway {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:37.068283 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-5bb62ad1-144d-44b4-b80c-a80e0918b9cb service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2009268 May 31 11:59:37.189976 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:59:34Z', 'revision_number': 5, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:37.194298 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4ff4d608-46e2-4933-99df-04c71bc5fc0d None None] The sync data for ids ['4c61aefe-0df0-474e-8c1a-3fa776e16bcc'] in dv-cinder-791239-4-iscsi is: [{'id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'name': 'tempest-TestMinimumBasicScenario-774559787-router', 'tenant_id': 'c8035d3733414b75bde96faf369f72d2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203'}], 'enable_snat': True}, 'gw_port_id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:56:10Z', 'updated_at': '2021-05-31T11:59:34Z', 'revision_number': 5, 'project_id': 'c8035d3733414b75bde96faf369f72d2', 'gw_port': {'id': '4ccc4832-093a-44f0-a0bb-60f43b576309', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:2e:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4c61aefe-0df0-474e-8c1a-3fa776e16bcc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 221, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.203', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:56:11Z', 'updated_at': '2021-05-31T11:56:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:37.294364 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d301ee22-449d-4c7c-b5a4-ed295bf8b40e req-65453ff3-b907-4c4d-9265-35756dab6455 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1736391 May 31 11:59:37.438331 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:37.455352 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e179b95b-9470-493c-9211-d09b560bb0c5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:37.604068 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Pushing event deleted for resources: {'Port': ['ID=4ccc4832-093a-44f0-a0bb-60f43b576309,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:37.653916 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-7e9c19b9-2b88-4466-add4-e2d682d56f2e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:37.655893 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-7e9c19b9-2b88-4466-add4-e2d682d56f2e service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 1097 time: 0.1849773 May 31 11:59:37.677014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.226s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:37.732732 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:37.813950 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-6c9d0ac5-0838-4435-acbb-5786bb855f34 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:37.815298 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-6c9d0ac5-0838-4435-acbb-5786bb855f34 service neutron] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 912 time: 0.1507635 May 31 11:59:37.904718 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-2e6db606-5784-4e42-ae49-4f67b522a1d2 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=6bd1051c-af78-4616-88bd-002db4d1b521 HTTP/1.1" status: 200 len: 217 time: 0.0825889 May 31 11:59:37.919945 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca0e8a75-923a-4510-9231-efbe52f8f808 req-a313b8ea-65b7-464b-b5e4-0a5e8d91449b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:37.926176 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ca0e8a75-923a-4510-9231-efbe52f8f808 req-a313b8ea-65b7-464b-b5e4-0a5e8d91449b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 944 time: 0.3246441 May 31 11:59:37.926176 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:37.926176 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:37.979313 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-fe50132e-efb6-44da-b01f-cb12f280e9b7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:37.989314 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-fe50132e-efb6-44da-b01f-cb12f280e9b7 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0693471 May 31 11:59:37.991545 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-ca0e8a75-923a-4510-9231-efbe52f8f808 req-7ffd3ead-142b-45b0-bd4a-39fb5a34c0d2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:security_group {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 11:59:38.004542 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-ca0e8a75-923a-4510-9231-efbe52f8f808 req-7ffd3ead-142b-45b0-bd4a-39fb5a34c0d2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:security_group {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 11:59:38.008011 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ca0e8a75-923a-4510-9231-efbe52f8f808 req-7ffd3ead-142b-45b0-bd4a-39fb5a34c0d2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0639119 May 31 11:59:38.097610 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.098306 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 4c61aefe-0df0-474e-8c1a-3fa776e16bcc {{(pid=99039) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 11:59:38.119128 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.120360 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-bd884011-3c2a-4c28-92b4-b590c8197ae2 admin admin] 10.222.0.48 "DELETE /v2.0/routers/4c61aefe-0df0-474e-8c1a-3fa776e16bcc HTTP/1.1" status: 204 len: 168 time: 2.3825557 May 31 11:59:38.122887 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36654) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:38.167301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-bd4e8791-3a1e-474d-9616-3699d7a8b96b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:38.169826 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-bd4e8791-3a1e-474d-9616-3699d7a8b96b service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1819079 May 31 11:59:38.189552 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Deleting subnet c01a063a-2236-4017-91be-4828f1866f28 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 11:59:38.192833 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.255680 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.341944 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-896698e4-b63c-4406-9b6d-d22b0234f0d1 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1530097 May 31 11:59:38.389521 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.479442 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d961555c-6b90-481f-bd7f-ccbb6a0cc1f2 req-da38f6c9-d11c-4528-9193-65713126f2cf service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1398935 May 31 11:59:38.578934 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36660) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:38.685486 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 4ccc4832-093a-44f0-a0bb-60f43b576309 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:38.726975 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.748350 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:38.750875 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 4ccc4832-093a-44f0-a0bb-60f43b576309 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:38.751321 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 4ccc4832-093a-44f0-a0bb-60f43b576309 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:38.751815 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 5c78fca6-f530-48c3-8190-890cb005d8f5 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:38.756079 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Usage tracker for resource:security_group_rule and tenant:dd07205fe9344efe870552d8a034f2b4 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:38.759251 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:38.770398 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:38.796009 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 5c78fca6-f530-48c3-8190-890cb005d8f5 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:38.796503 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 5c78fca6-f530-48c3-8190-890cb005d8f5 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:38.857628 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.866633 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.871624 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:38.908026 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Pushing event deleted for resources: {'Subnet': ['ID=c01a063a-2236-4017-91be-4828f1866f28,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:38.909995 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.912503 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:38.914961 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e73d88d7-fdfb-4434-9098-904bed249233 None None] Network 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:38.926896 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] 10.222.0.48 "DELETE /v2.0/subnets/c01a063a-2236-4017-91be-4828f1866f28 HTTP/1.1" status: 204 len: 168 time: 0.7926605 May 31 11:59:38.926896 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36668) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:38.952950 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:38.965594 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5bb90993-094a-490a-aaa5-43c98a6b7b63 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:39.017284 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=dd07205fe9344efe870552d8a034f2b4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4342487 May 31 11:59:39.017284 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ea84f475-0330-402b-b484-d93fb2f94631,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:39.021373 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:39.022973 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36672) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:39.035127 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.074023 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.126685 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-42b82840-babd-40b1-bfe2-67aa9f969319 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.127017 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Deleting port c495e266-af80-4662-84fa-21d967aef128 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:39.127503 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.157885 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-42b82840-babd-40b1-bfe2-67aa9f969319 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.221602 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] DHCP port deletion for 87442cb7-bfc4-4bf2-9d82-09a2c5a5344f request from dv-cinder-791239-4-iscsi {{(pid=99041) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 31 11:59:39.289538 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] For port c495e266-af80-4662-84fa-21d967aef128, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=c495e266-af80-4662-84fa-21d967aef128,segment=NetworkSegment(0eba2510-55a0-4815-bcb1-2dc142aabf11),segment_id=0eba2510-55a0-4815-bcb1-2dc142aabf11)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:39.290147 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.300738 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] Deleting port c495e266-af80-4662-84fa-21d967aef128 {{(pid=99041) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 11:59:39.301942 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.357357 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-42b82840-babd-40b1-bfe2-67aa9f969319 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.365102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3c3fe967-aa76-4a5e-a9a2-57a57e4a10a2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:39.382946 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-42b82840-babd-40b1-bfe2-67aa9f969319 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ea84f475-0330-402b-b484-d93fb2f94631,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:39.385887 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-42b82840-babd-40b1-bfe2-67aa9f969319 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:39.388093 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-42b82840-babd-40b1-bfe2-67aa9f969319 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.388883 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-42b82840-babd-40b1-bfe2-67aa9f969319 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/ea84f475-0330-402b-b484-d93fb2f94631 HTTP/1.1" status: 204 len: 168 time: 0.3634641 May 31 11:59:39.436570 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Calling delete_port for c495e266-af80-4662-84fa-21d967aef128 owned by network:dhcp {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:39.444678 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] For port c495e266-af80-4662-84fa-21d967aef128, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=c495e266-af80-4662-84fa-21d967aef128,segment=NetworkSegment(0eba2510-55a0-4815-bcb1-2dc142aabf11),segment_id=0eba2510-55a0-4815-bcb1-2dc142aabf11)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:39.444678 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.465469 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f8336c0-c4de-47f7-82e0-4a6189f9fd97 req-2232555d-52b0-4c7e-a843-eb1113ec0f9d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:39.466984 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1f8336c0-c4de-47f7-82e0-4a6189f9fd97 req-2232555d-52b0-4c7e-a843-eb1113ec0f9d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 944 time: 0.3232970 May 31 11:59:39.526098 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1f8336c0-c4de-47f7-82e0-4a6189f9fd97 req-e5acab70-5ba8-40be-b7a3-b4459f70c537 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0515175 May 31 11:59:39.606276 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] Calling delete_port for c495e266-af80-4662-84fa-21d967aef128 owned by network:dhcp {{(pid=99041) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 11:59:39.667568 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.671520 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a9c73a1f-6637-4a4c-bc57-746b9897d775 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:39.673527 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.684638 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:39.714867 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Pushing event deleted for resources: {'Port': ['ID=c495e266-af80-4662-84fa-21d967aef128,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:39.718356 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:39.719561 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] Pushing event deleted for resources: {'Port': ['ID=c495e266-af80-4662-84fa-21d967aef128,revision_number=None']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:39.721543 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] No DVR routers for this DVR port c495e266-af80-4662-84fa-21d967aef128 on host dv-cinder-791239-4-iscsi {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:59:39.723970 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:39.728636 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] No DVR routers for this DVR port c495e266-af80-4662-84fa-21d967aef128 on host dv-cinder-791239-4-iscsi {{(pid=99041) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 11:59:39.845071 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.858783 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['87442cb7-bfc4-4bf2-9d82-09a2c5a5344f']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:39.873913 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '0eba2510-55a0-4815-bcb1-2dc142aabf11') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:39.874502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '0eba2510-55a0-4815-bcb1-2dc142aabf11') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:39.889242 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.910673 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.916926 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Releasing vlan 376 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 11:59:39.917333 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 11:59:39.931631 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 11:59:40.232731 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.241593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6dd3aca2-045e-4681-896f-9d914d8c560c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:40.250741 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.260988 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Pushing event deleted for resources: {'Network': ['ID=87442cb7-bfc4-4bf2-9d82-09a2c5a5344f,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:40.263813 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:40.265743 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] 10.222.0.48 "DELETE /v2.0/networks/87442cb7-bfc4-4bf2-9d82-09a2c5a5344f HTTP/1.1" status: 204 len: 168 time: 1.3455541 May 31 11:59:40.345987 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36690) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:40.364246 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.379168 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:40.382969 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Usage tracker for resource:security_group_rule and tenant:90e94a05af644bbfa18a2da69eff1d92 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:40.390446 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:40.392358 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:40.481833 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.549181 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 6bd1051c-af78-4616-88bd-002db4d1b521 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 11:59:40.627903 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.670337 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c552f173-22be-49e3-9245-45533f584ee7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:40.698344 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device c495e266-af80-4662-84fa-21d967aef128 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 11:59:40.706234 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=90e94a05af644bbfa18a2da69eff1d92&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3583484 May 31 11:59:40.707020 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36696) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:40.747493 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ba1e53d7-2f11-4484-8c37-420926a422eb,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:40.755220 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6639a87e-e1bd-4508-aa65-5e696518ebf6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:40.778235 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36700) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:40.778395 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: c495e266-af80-4662-84fa-21d967aef128 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 11:59:40.778838 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device c495e266-af80-4662-84fa-21d967aef128 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 11:59:40.801681 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-aed20358-eb17-483b-90fd-ef96ab173b20 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.803186 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 6bd1051c-af78-4616-88bd-002db4d1b521, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6bd1051c-af78-4616-88bd-002db4d1b521,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:40.813221 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db88a255-20f3-495c-9ac2-d93b8dd3a0ac req-95610fb4-4c57-4962-813a-cb320171e651 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:40.815193 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-db88a255-20f3-495c-9ac2-d93b8dd3a0ac req-95610fb4-4c57-4962-813a-cb320171e651 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 944 time: 0.1548736 May 31 11:59:40.827562 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '6bd1051c-af78-4616-88bd-002db4d1b521', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'port_id': '6bd1051c-af78-4616-88bd-002db4d1b521', 'mac_address': 'fa:16:3e:b6:5c:90', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 363, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.14'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 11:59:40.835548 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-aed20358-eb17-483b-90fd-ef96ab173b20 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.888234 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-02acefa5-65fe-4b18-b3e9-ba336bfa4b49 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=39ae37364a184ae7ae341ea7b2847c35&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1053038 May 31 11:59:40.888619 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36704) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:40.910391 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-db88a255-20f3-495c-9ac2-d93b8dd3a0ac req-692c8a92-e0d1-4cf1-b3ee-fcd995e5f06b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0819311 May 31 11:59:40.919996 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.931511 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 6bd1051c-af78-4616-88bd-002db4d1b521 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 11:59:40.951640 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 6bd1051c-af78-4616-88bd-002db4d1b521, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6bd1051c-af78-4616-88bd-002db4d1b521,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:40.956550 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6501b731-9a7f-485e-8823-6b49ecd94894 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:40.986786 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6501b731-9a7f-485e-8823-6b49ecd94894 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:41.008227 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-aed20358-eb17-483b-90fd-ef96ab173b20 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:41.018914 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-72912336-70dc-4e0b-a101-fd8d057dae84 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:41.043333 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aed20358-eb17-483b-90fd-ef96ab173b20 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ba1e53d7-2f11-4484-8c37-420926a422eb,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:41.047645 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-aed20358-eb17-483b-90fd-ef96ab173b20 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:41.047787 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:41.050427 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:41.056118 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-aed20358-eb17-483b-90fd-ef96ab173b20 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:41.057303 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-aed20358-eb17-483b-90fd-ef96ab173b20 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/ba1e53d7-2f11-4484-8c37-420926a422eb HTTP/1.1" status: 204 len: 168 time: 0.3492737 May 31 11:59:41.143157 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=6bd1051c-af78-4616-88bd-002db4d1b521,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:41.146505 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:41.192804 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6501b731-9a7f-485e-8823-6b49ecd94894 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:41.197760 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-aec051dd-f0ab-4def-bede-b1267cc84f29 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:41.213090 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6501b731-9a7f-485e-8823-6b49ecd94894 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7392633c-3cf7-4366-bb3e-fd78503c0a30,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:41.216737 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6501b731-9a7f-485e-8823-6b49ecd94894 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:41.269548 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['6bd1051c-af78-4616-88bd-002db4d1b521'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 11:59:41.346203 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6501b731-9a7f-485e-8823-6b49ecd94894 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:41.349711 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-6501b731-9a7f-485e-8823-6b49ecd94894 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/7392633c-3cf7-4366-bb3e-fd78503c0a30 HTTP/1.1" status: 204 len: 168 time: 0.4572442 May 31 11:59:42.112813 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4470f3b-f9ff-4166-9100-feb0ad1fc2f9 req-1ee2597f-d7c4-4f5a-bf0a-e01b3107508c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:42.114358 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d4470f3b-f9ff-4166-9100-feb0ad1fc2f9 req-1ee2597f-d7c4-4f5a-bf0a-e01b3107508c tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 945 time: 0.0943811 May 31 11:59:42.149623 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36736) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:42.186924 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d4470f3b-f9ff-4166-9100-feb0ad1fc2f9 req-12066994-41ba-4970-854e-be4b59684e46 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0647233 May 31 11:59:42.223537 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-8f0ca00b-44f3-4e19-b053-ab98d04777f9 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=c8035d3733414b75bde96faf369f72d2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0760498 May 31 11:59:42.225995 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36740) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:42.295508 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-397bb3eb-9a31-4dca-8ce7-c6e347b603c3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.326360 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-397bb3eb-9a31-4dca-8ce7-c6e347b603c3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.496128 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-397bb3eb-9a31-4dca-8ce7-c6e347b603c3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.500464 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-fb2177d9-3cab-4bce-9159-c2825fe090a6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:42.515569 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-397bb3eb-9a31-4dca-8ce7-c6e347b603c3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=78fe6823-3c2b-4873-b2f4-33f35793a5dd,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:42.519537 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-397bb3eb-9a31-4dca-8ce7-c6e347b603c3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:42.520338 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-397bb3eb-9a31-4dca-8ce7-c6e347b603c3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.526336 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-397bb3eb-9a31-4dca-8ce7-c6e347b603c3 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/78fe6823-3c2b-4873-b2f4-33f35793a5dd HTTP/1.1" status: 204 len: 168 time: 0.2943423 May 31 11:59:42.556670 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 6bd1051c-af78-4616-88bd-002db4d1b521 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:42.581695 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36752) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:42.711720 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 6bd1051c-af78-4616-88bd-002db4d1b521 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:42.716942 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 6bd1051c-af78-4616-88bd-002db4d1b521 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:42.717326 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.729608 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-180171188-network', 'tenant_id': '1225cc76a5a0424ab646043550b5a0d8'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:42.734908 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:42.739124 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Usage tracker for resource:network and tenant:1225cc76a5a0424ab646043550b5a0d8 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:42.742136 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:42.748074 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:42.814572 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Made reservation on behalf of 1225cc76a5a0424ab646043550b5a0d8 for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:42.817156 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.831454 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.842489 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.846861 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:42.850654 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Usage tracker for resource:security_group_rule and tenant:1225cc76a5a0424ab646043550b5a0d8 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:42.853504 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:42.860796 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:42.871478 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6bd1051c-af78-4616-88bd-002db4d1b521'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:42.873768 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-31781ecf-7b22-49ce-a88c-0fa54124b08c" -d '{"events": [{"server_uuid": "dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca", "name": "network-vif-plugged", "status": "completed", "tag": "6bd1051c-af78-4616-88bd-002db4d1b521"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:42.887869 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 6bd1051c-af78-4616-88bd-002db4d1b521, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6bd1051c-af78-4616-88bd-002db4d1b521,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:42.929800 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 11:59:42 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-e5f1495a-f72a-429a-b135-36995ec43d9e x-openstack-request-id: req-e5f1495a-f72a-429a-b135-36995ec43d9e {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:42.930321 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca", "name": "network-vif-plugged", "status": "completed", "tag": "6bd1051c-af78-4616-88bd-002db4d1b521", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:42.930576 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-e5f1495a-f72a-429a-b135-36995ec43d9e {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:42.931130 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6bd1051c-af78-4616-88bd-002db4d1b521', 'code': 200} May 31 11:59:42.960748 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.985082 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:42.988142 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-385c39f8-416e-4452-bc87-8827f2540a21 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:43.067557 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=6bd1051c-af78-4616-88bd-002db4d1b521,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:43.070764 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:43.102032 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:43.107094 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-afb9f8ee-2263-4281-993b-ca0a34866a3a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:43.135164 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] vlan segment allocate from pool success with {'vlan_id': 375, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:59:43.151813 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=56fa3b08-f1c3-4248-9175-95d2ebb4e120,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:43.155380 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:43.156675 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:43.157006 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Added segment ce07ff9d-e078-4661-b74a-f01d070a87f3 of type vlan for network 59773d4e-a35c-4ba4-8bef-ff4d4070804c May 31 11:59:43.187276 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:43.352193 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:43.361696 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:43.362281 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:43.369988 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:43.518723 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Pushing event updated for resources: {'Network': ['ID=59773d4e-a35c-4ba4-8bef-ff4d4070804c,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:43.526136 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:43.530374 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 59773d4e-a35c-4ba4-8bef-ff4d4070804c details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:59:43.565722 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d933fe0d-e137-496e-bc9a-6617d49861f3 req-6b42dd3b-3d66-4868-85bf-637a4536d0fd tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:43.567146 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d933fe0d-e137-496e-bc9a-6617d49861f3 req-6b42dd3b-3d66-4868-85bf-637a4536d0fd tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 946 time: 0.2575800 May 31 11:59:43.577061 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 0.9941988 May 31 11:59:43.580344 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36782) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:43.587074 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Request body: {'subnet': {'network_id': '59773d4e-a35c-4ba4-8bef-ff4d4070804c', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-180171188-subnet', 'tenant_id': '1225cc76a5a0424ab646043550b5a0d8', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:43.592321 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:43.604494 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Usage tracker for resource:subnet and tenant:1225cc76a5a0424ab646043550b5a0d8 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:43.607291 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e79e586c-27d4-4f29-86f2-a643603c0549 req-56d31fe3-f989-43b1-9985-acdc06a636df service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:43.608562 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:43.608748 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e79e586c-27d4-4f29-86f2-a643603c0549 req-56d31fe3-f989-43b1-9985-acdc06a636df service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 1099 time: 0.2577233 May 31 11:59:43.617151 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:43.647548 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d933fe0d-e137-496e-bc9a-6617d49861f3 req-53a58ed5-a5ee-4e21-9d28-49ecaefa0083 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0691319 May 31 11:59:43.672421 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e79e586c-27d4-4f29-86f2-a643603c0549 req-574c1fda-cb5d-4cc9-8506-c50ab21d5bb4 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0549517 May 31 11:59:43.690084 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Made reservation on behalf of 1225cc76a5a0424ab646043550b5a0d8 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:43.692630 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:43.793392 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36790) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:43.904996 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-1197050931-network', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:43.910249 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:43.914581 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Usage tracker for resource:network and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:43.917522 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:43.924462 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:44.044430 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Made reservation on behalf of bbcb901221a14602a1f2f8b198287636 for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:44.047750 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.063856 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.078940 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:44.082982 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Usage tracker for resource:security_group_rule and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:44.085890 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:44.094430 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:44.184847 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.266957 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.271122 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-42b82840-babd-40b1-bfe2-67aa9f969319 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:44.302562 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] vlan segment allocate from pool success with {'vlan_id': 397, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:59:44.312117 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=31834e32-c3e8-4d1f-acda-99841f2d99c8,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:44.315951 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:44.318502 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.319825 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Added segment edd9824c-4e69-4a31-bd8c-e501618fb7bb of type vlan for network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f May 31 11:59:44.342941 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.349392 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d2e5af79-0c25-42e2-91c7-4d5bbce1f255 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:44.349596 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.442352 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Pushing event updated for resources: {'Subnet': ['ID=ad6c0027-9f7a-4d64-a9a0-1449192c5ee4,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:44.445794 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:44.448577 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Checking segment: {'id': 'ce07ff9d-e078-4661-b74a-f01d070a87f3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 375, 'network_id': '59773d4e-a35c-4ba4-8bef-ff4d4070804c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:44.448767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Network hostable DHCP agents. Network: 59773d4e-a35c-4ba4-8bef-ff4d4070804c, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:59:44.473120 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.479358 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:44.479358 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.479583 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c4c237ab-35a9-48f4-8164-81846265db76 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:44.524101 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Network 59773d4e-a35c-4ba4-8bef-ff4d4070804c is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:59:44.581443 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 59773d4e-a35c-4ba4-8bef-ff4d4070804c) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:44.592628 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-021de50e-39ce-4d57-b2e0-8ee9d5eccaea None None] Network 59773d4e-a35c-4ba4-8bef-ff4d4070804c requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:44.593164 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:44.593334 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:44.680470 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Pushing event updated for resources: {'Network': ['ID=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:44.683097 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.204s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:44.692902 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:59:44.709639 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c8512689-6650-4b71-b991-5b9351108818 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.1253583 May 31 11:59:44.811312 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 1.0142689 May 31 11:59:44.812310 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36804) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:44.818972 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Request body: {'subnet': {'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-1197050931-subnet', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:44.828722 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:44.836889 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Usage tracker for resource:subnet and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:44.840942 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:44.851262 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:44.872273 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5f7caaaf-0557-43d2-ad09-a8a751ca6219 req-8dd44884-37d5-40d2-a4f6-fed16a816ce7 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:44.882244 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5f7caaaf-0557-43d2-ad09-a8a751ca6219 req-8dd44884-37d5-40d2-a4f6-fed16a816ce7 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 946 time: 0.1224411 May 31 11:59:44.914914 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-47f26818-d217-4e74-bdd1-b5c576f99535 None None] Network 59773d4e-a35c-4ba4-8bef-ff4d4070804c requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:44.924481 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5f7caaaf-0557-43d2-ad09-a8a751ca6219 req-ff9491aa-6d53-4f7a-afd2-793f1499f836 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0413129 May 31 11:59:44.954426 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Made reservation on behalf of bbcb901221a14602a1f2f8b198287636 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:44.954426 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.448090 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.452756 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:45.522973 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Pushing event updated for resources: {'Subnet': ['ID=3a19e015-3084-4732-bb65-ea124df3118f,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:45.527583 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:45.535293 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36822) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:45.538262 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Checking segment: {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:45.538618 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Network hostable DHCP agents. Network: fc7e08b1-448f-4bc6-8c22-cdf76e730c2f, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:59:45.540662 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-1908593432-network', 'tenant_id': 'e080a39f16314f14b44474063dd46cff'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:45.549274 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:45.554475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Usage tracker for resource:network and tenant:e080a39f16314f14b44474063dd46cff is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:45.557945 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:45.564304 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:45.655661 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:59:45.669226 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Made reservation on behalf of e080a39f16314f14b44474063dd46cff for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:45.671576 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.684374 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.701486 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:45.705933 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Usage tracker for resource:security_group_rule and tenant:e080a39f16314f14b44474063dd46cff is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:45.708677 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:45.714350 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:45.724338 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, fc7e08b1-448f-4bc6-8c22-cdf76e730c2f) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:45.735976 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e2cbb11d-f759-4ee8-ac5d-5837cf7b23fe None None] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:45.748857 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:45.749598 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.801754 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.839533 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.0261056 May 31 11:59:45.842078 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36826) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:45.847201 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:45.855039 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:45.859020 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Usage tracker for resource:router and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:45.862450 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:45.868116 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:45.907083 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.911193 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-397bb3eb-9a31-4dca-8ce7-c6e347b603c3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:45.938797 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] vlan segment allocate from pool success with {'vlan_id': 369, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:59:45.952058 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=25fb30a6-1542-40de-9b70-1bb2e82cb2c7,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:45.957515 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:45.958387 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.958824 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Added segment 59945c0f-16e2-4afd-b32b-ccf004cf3f18 of type vlan for network 49520081-8b4f-454e-b744-1f256a1202d2 May 31 11:59:45.974000 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 11:59:45.975187 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.991301 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.992805 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:45.995518 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.012230 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.158093 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Allocated IP 172.20.0.2 (fc7e08b1-448f-4bc6-8c22-cdf76e730c2f/3a19e015-3084-4732-bb65-ea124df3118f/28d2e39a-c647-40fa-a4f8-8a846693f680) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:46.162271 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-524ec4e9-e9bd-4260-823a-b1ff243ca700 req-3ad4b948-ec1c-4bc5-a9b0-8df89fb165aa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:46.166608 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-524ec4e9-e9bd-4260-823a-b1ff243ca700 req-3ad4b948-ec1c-4bc5-a9b0-8df89fb165aa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 946 time: 0.0948362 May 31 11:59:46.169405 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.173399 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:46.174530 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.192461 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9c0f1c2d-190b-42a3-8e56-93b2b6123087 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:46.318891 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-524ec4e9-e9bd-4260-823a-b1ff243ca700 req-59560690-77b4-45f1-8305-11b2c12b705e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1481996 May 31 11:59:46.319904 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Pushing event updated for resources: {'Network': ['ID=49520081-8b4f-454e-b744-1f256a1202d2,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:46.322833 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:46.347980 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 49520081-8b4f-454e-b744-1f256a1202d2 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:59:46.388752 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.8523455 May 31 11:59:46.389966 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36836) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:46.398749 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Request body: {'subnet': {'network_id': '49520081-8b4f-454e-b744-1f256a1202d2', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-1908593432-subnet', 'tenant_id': 'e080a39f16314f14b44474063dd46cff', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:46.403430 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:46.408311 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Usage tracker for resource:subnet and tenant:e080a39f16314f14b44474063dd46cff is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:46.411174 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:46.420936 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:46.434024 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.474844 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.489012 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Transition to ACTIVE for port object 28d2e39a-c647-40fa-a4f8-8a846693f680 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:46.499029 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Made reservation on behalf of e080a39f16314f14b44474063dd46cff for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:46.501952 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.512917 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Transition to ACTIVE for port object 28d2e39a-c647-40fa-a4f8-8a846693f680 will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:46.622088 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.626275 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:46.641101 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3782855-419f-4b25-876c-88f9b2cbf42e req-576096f6-214c-47e6-a1ba-3855ce76069a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:46.645252 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c3782855-419f-4b25-876c-88f9b2cbf42e req-576096f6-214c-47e6-a1ba-3855ce76069a tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 946 time: 0.2119749 May 31 11:59:46.654052 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 36848) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:46.662608 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Request body: {'floatingip': {'port_id': '6bd1051c-af78-4616-88bd-002db4d1b521'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:46.691967 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:46.695007 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:46.698452 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:46.735455 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Pushing event updated for resources: {'Port': ['ID=28d2e39a-c647-40fa-a4f8-8a846693f680,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:46.742482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:46.771017 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.800959 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Attempting to bind port 28d2e39a-c647-40fa-a4f8-8a846693f680 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:59:46.801417 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Attempting to bind port 28d2e39a-c647-40fa-a4f8-8a846693f680 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:59:46.805043 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Attempting to bind port 28d2e39a-c647-40fa-a4f8-8a846693f680 on network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:59:46.807283 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 59, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:59:46.807659 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Checking segment: {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:46.808336 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.809014 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Bound using segment: {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:59:46.809718 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Bound port: 28d2e39a-c647-40fa-a4f8-8a846693f680, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:59:46.810262 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:46.904557 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] For port 28d2e39a-c647-40fa-a4f8-8a846693f680, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:59:46.910069 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] For port 28d2e39a-c647-40fa-a4f8-8a846693f680, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=28d2e39a-c647-40fa-a4f8-8a846693f680,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:59:47.004867 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:47.008688 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:47.170287 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Pushing event updated for resources: {'Port': ['ID=28d2e39a-c647-40fa-a4f8-8a846693f680,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:47.188310 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.172s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:47.242923 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:47.284423 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:47.478433 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:47.523548 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Allocated IP 172.27.1.158 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:47.541721 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:47.568021 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:47.581026 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:47.717228 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Floating IP 35e7f9d3-2578-4db6-a21c-2bc73f629dc0 associated. External IP: 172.27.1.248, port: 6bd1051c-af78-4616-88bd-002db4d1b521. May 31 11:59:47.745487 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Pushing event updated for resources: {'Subnet': ['ID=3232ed29-0c0e-4db0-a48b-7e3bd671f4ff,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:47.759808 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:47.773201 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Checking segment: {'id': '59945c0f-16e2-4afd-b32b-ccf004cf3f18', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 369, 'network_id': '49520081-8b4f-454e-b744-1f256a1202d2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:47.773551 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Network hostable DHCP agents. Network: 49520081-8b4f-454e-b744-1f256a1202d2, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:59:47.798041 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:47.958984 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Network 49520081-8b4f-454e-b744-1f256a1202d2 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:59:47.981960 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:47.987788 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-acecc810-6a7e-4121-83c3-3ce7274d488b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:48.049616 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:48.052786 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 49520081-8b4f-454e-b744-1f256a1202d2) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:48.054599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:48.063240 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:48.066390 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-205a03fe-29ed-4647-ba67-fcaf1097a2d3 None None] Network 49520081-8b4f-454e-b744-1f256a1202d2 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:48.107975 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Pushing event updated for resources: {'Port': ['ID=a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:48.117106 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:48.120406 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:48.120996 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:48.200404 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Router 6c5aae18-4faa-4947-af35-4078d428c2b9 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:59:48.250454 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d488c1e8-9108-44d2-83e9-770bea2dc54e None None] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:48.302296 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-dc1162f5-584b-4cdc-bbbe-a1ce33ffc3bd admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 1.9107134 May 31 11:59:48.414925 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:48.419783 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:48.431222 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] Sync routers for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:48.452608 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-501b567a-1738-4f66-8f06-2d3d55ae9f44 None None] Network 49520081-8b4f-454e-b744-1f256a1202d2 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:48.491439 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-d4f4c67f-6ae2-4cc8-9016-50da589f7b8d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "PUT /v2.0/floatingips/35e7f9d3-2578-4db6-a21c-2bc73f629dc0 HTTP/1.1" status: 200 len: 992 time: 1.8357935 May 31 11:59:48.492196 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'tag': '6bd1051c-af78-4616-88bd-002db4d1b521'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 11:59:48.495181 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-1a564c5d-9ad5-4588-a0cd-622db7461fbe" -d '{"events": [{"name": "network-changed", "server_uuid": "dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca", "tag": "6bd1051c-af78-4616-88bd-002db4d1b521"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 11:59:48.553040 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:48.553798 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:48.647965 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-131577b7-1327-45fd-8e4b-5eff9d33eb1c None None] Provisioning for port 28d2e39a-c647-40fa-a4f8-8a846693f680 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:48.710863 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:48.788800 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 11:59:48 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-0d1d41f0-f841-476a-a83b-f9e2322e6cf6 x-openstack-request-id: req-0d1d41f0-f841-476a-a83b-f9e2322e6cf6 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 11:59:48.789521 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca", "tag": "6bd1051c-af78-4616-88bd-002db4d1b521", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 11:59:48.789521 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-0d1d41f0-f841-476a-a83b-f9e2322e6cf6 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 11:59:48.789872 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'tag': '6bd1051c-af78-4616-88bd-002db4d1b521', 'status': 'completed', 'code': 200} May 31 11:59:48.839453 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 28d2e39a-c647-40fa-a4f8-8a846693f680 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:48.906288 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-f8e6bfe6-a6e5-4af1-8925-1602686b4019 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:48.908670 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-f8e6bfe6-a6e5-4af1-8925-1602686b4019 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=b866e6eab1b04406a7280f91698983ce&device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 1099 time: 0.0888169 May 31 11:59:49.080531 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:49.131301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-85ecc159-0f8b-45a9-965f-96b8dbb9c057 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:49.132805 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-85ecc159-0f8b-45a9-965f-96b8dbb9c057 service neutron] 10.222.0.47 "GET /v2.0/networks?id=36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 200 len: 912 time: 0.2180686 May 31 11:59:49.252614 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:49.262146 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-c9d203cf-fcba-4bf0-878f-cc8175cd3053 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=6bd1051c-af78-4616-88bd-002db4d1b521 HTTP/1.1" status: 200 len: 1000 time: 0.1139600 May 31 11:59:49.329684 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-1731a3c7-a001-4386-bf1b-a04ab240e8af service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:49.331133 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-1731a3c7-a001-4386-bf1b-a04ab240e8af service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 200 len: 850 time: 0.0621667 May 31 11:59:49.374445 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 28d2e39a-c647-40fa-a4f8-8a846693f680 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:49.420292 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-b468318f-9189-44d8-9945-14a1b73e3b21 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:49.421869 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-b468318f-9189-44d8-9945-14a1b73e3b21 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=36cce22a-502c-464d-a9ac-50f2bc247e07&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0844650 May 31 11:59:49.463185 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 28d2e39a-c647-40fa-a4f8-8a846693f680 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:49.463926 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:49.586941 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Router 0da52ba7-9067-4610-8053-ced8a908f89b is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99039) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 11:59:49.636708 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-900a10d5-c8a1-46a9-b1c3-4d8cba0f5f6d service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2030561 May 31 11:59:49.761265 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:49.788338 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:49.799122 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Sync routers for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:49.810520 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:49.811040 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:49.830319 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 28d2e39a-c647-40fa-a4f8-8a846693f680, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=28d2e39a-c647-40fa-a4f8-8a846693f680,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:49.835614 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1a564c5d-9ad5-4588-a0cd-622db7461fbe req-75febd60-81f9-49bc-b768-10747afbe7e2 service neutron] 10.222.0.47 "GET /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1909304 May 31 11:59:49.847440 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-1e24e627-8892-462e-a018-70781d15dd08 None None] Provisioning complete for port 28d2e39a-c647-40fa-a4f8-8a846693f680 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:49.847844 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1e24e627-8892-462e-a018-70781d15dd08 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:49.899966 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:49.905554 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:49.921424 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:49.923761 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 921 time: 4.0806072 May 31 11:59:49.926462 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36886) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:49.979669 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '35e7f9d3-2578-4db6-a21c-2bc73f629dc0', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'floating_ip_address': '172.27.1.248', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'port_id': '6bd1051c-af78-4616-88bd-002db4d1b521', 'fixed_ip_address': '172.20.0.14', 'status': 'DOWN', 'standard_attr_id': 460, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'mac_address': 'fa:16:3e:b6:5c:90', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:59:27Z', 'updated_at': '2021-05-31T11:59:47Z', 'revision_number': 1, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:49.992065 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] Checking router: 6c5aae18-4faa-4947-af35-4078d428c2b9 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:50.021784 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:50.022623 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:50.053396 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:50.113412 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=28d2e39a-c647-40fa-a4f8-8a846693f680,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:50.126164 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.220s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:50.346075 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:50.379627 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:48Z', 'revision_number': 3, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:47Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:50.383475 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Checking router: 0da52ba7-9067-4610-8053-ced8a908f89b for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:50.383827 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Updating router 0da52ba7-9067-4610-8053-ced8a908f89b port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:59:50.445707 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:50.695633 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:50.725639 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 28d2e39a-c647-40fa-a4f8-8a846693f680 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:50.842267 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 28d2e39a-c647-40fa-a4f8-8a846693f680 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:50.842673 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:50.849279 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:50.862162 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Transition to ACTIVE for port object a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:50.877816 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '35e7f9d3-2578-4db6-a21c-2bc73f629dc0', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'floating_ip_address': '172.27.1.248', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'port_id': '6bd1051c-af78-4616-88bd-002db4d1b521', 'fixed_ip_address': '172.20.0.14', 'status': 'DOWN', 'standard_attr_id': 460, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'mac_address': 'fa:16:3e:b6:5c:90', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:59:27Z', 'updated_at': '2021-05-31T11:59:47Z', 'revision_number': 1, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:50.895727 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2626fcae-ba87-475b-b5c7-cdcae04e4503 None None] The sync data for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi is: [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '35e7f9d3-2578-4db6-a21c-2bc73f629dc0', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'floating_ip_address': '172.27.1.248', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'port_id': '6bd1051c-af78-4616-88bd-002db4d1b521', 'fixed_ip_address': '172.20.0.14', 'status': 'DOWN', 'standard_attr_id': 460, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'mac_address': 'fa:16:3e:b6:5c:90', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T11:59:27Z', 'updated_at': '2021-05-31T11:59:47Z', 'revision_number': 1, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:50.951115 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8bcbc199-5725-4737-ada1-9c48a49704b9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:50.988026 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:50.992587 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:51.068794 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Pushing event updated for resources: {'Port': ['ID=a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4,revision_number=2']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:51.077324 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.077723 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.078067 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.078397 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.078723 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.079162 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.079490 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.079810 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.080157 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.080475 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.080789 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.081203 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 11:59:51.088620 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Allocated IP 172.20.0.1 (fc7e08b1-448f-4bc6-8c22-cdf76e730c2f/3a19e015-3084-4732-bb65-ea124df3118f/824b42f8-92ee-4c2c-97e5-fdb7ab56aca3) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:51.094459 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:51.104859 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Attempting to bind port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:59:51.105230 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Attempting to bind port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:59:51.105621 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Attempting to bind port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:59:51.111910 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 59, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:59:51.116483 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:51.116483 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:51.116483 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:59:51.116483 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Bound port: a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:59:51.116483 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:51.176063 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e2dd620d-de81-4059-a471-9319aef0c3d8 None None] New status for floating IP 35e7f9d3-2578-4db6-a21c-2bc73f629dc0: ACTIVE {{(pid=99041) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 11:59:51.217683 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] For port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:59:51.223893 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] For port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:59:51.275780 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:51.291993 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Transition to ACTIVE for port object 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 will not be triggered until provisioned by entity DHCP. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:51.330421 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:51.334430 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:51.414431 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:51.415056 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:51.433275 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:51.437018 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:51.513722 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:51.517722 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:51.521188 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:51.558666 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Pushing event updated for resources: {'Port': ['ID=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:51.578731 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:51.589522 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:51.649345 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Pushing event updated for resources: {'Port': ['ID=a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:51.662196 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.328s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:51.695560 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:51.941244 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-4eb118d0-c3db-4709-a4d9-b170a54dca7f None None] Provisioning for port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:51.946258 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-4eb118d0-c3db-4709-a4d9-b170a54dca7f None None] Provisioning complete for port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:51.946857 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-4eb118d0-c3db-4709-a4d9-b170a54dca7f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:51.991866 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:52.018619 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:48Z', 'revision_number': 3, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:52.026259 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8bb22db0-265d-4503-8ad3-603fd426c37d None None] The sync data for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi is: [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:48Z', 'revision_number': 3, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:52.034532 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-4eb118d0-c3db-4709-a4d9-b170a54dca7f None None] Port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:59:52.271847 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:52.380001 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:52.414889 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-ec5f1fee-169b-417a-9b0b-3fb49d01ecb9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:52.502150 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Pushing event updated for resources: {'Port': ['ID=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:52.505217 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:52.511085 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:52.661302 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:52.662622 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:52.722037 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:52.861181 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:52.866779 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:52.867406 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:52.876486 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Router 0da52ba7-9067-4610-8053-ced8a908f89b has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 11:59:53.012621 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:53.026345 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5aeda07d-8a11-4b65-a023-4a67576674b4 None None] Provisioning complete for port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:53.026727 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5aeda07d-8a11-4b65-a023-4a67576674b4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:53.078743 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:53.095282 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5aeda07d-8a11-4b65-a023-4a67576674b4 None None] Port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 11:59:53.210667 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:53.213525 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5d0d9436-e352-4dfc-861c-5722119e5ef9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:53.250897 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:53.272234 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] 10.222.0.48 "PUT /v2.0/routers/0da52ba7-9067-4610-8053-ced8a908f89b/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.3445525 May 31 11:59:53.333611 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:53.336849 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:53.400349 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:53.529069 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-fe72b28b-3cb9-4de6-a0f9-dd20c1e3f1b1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:53.778148 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Sync routers for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:53.787423 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:53.788018 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:53.989395 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:54.189716 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36956) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:54.195478 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-1085702316-network', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:54.200337 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:54.204258 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Usage tracker for resource:network and tenant:00afe2f4c68c431385f2a4b6061b0f72 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:54.207248 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:54.213546 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:54.284049 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Made reservation on behalf of 00afe2f4c68c431385f2a4b6061b0f72 for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:54.287177 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.300712 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.310872 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:52Z', 'revision_number': 2, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:54.315975 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Checking router: 0da52ba7-9067-4610-8053-ced8a908f89b for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 11:59:54.316336 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Updating router 0da52ba7-9067-4610-8053-ced8a908f89b port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 11:59:54.324458 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:54.328423 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Usage tracker for resource:security_group_rule and tenant:00afe2f4c68c431385f2a4b6061b0f72 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:54.331832 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:54.338702 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:54.375259 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.430210 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.513169 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.518548 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f51dd096-4f38-41ed-a009-a4d3000d0a8c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:54.545294 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] vlan segment allocate from pool success with {'vlan_id': 365, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 11:59:54.554594 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=93a559f8-0bff-4177-ad11-64a4f77793cc,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:54.558157 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:54.560669 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.561012 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Added segment 29648ab6-8c7b-47d7-9f64-2b32a50ee3f7 of type vlan for network f1496a40-5ccd-4222-984b-dbffd9fdd474 May 31 11:59:54.586669 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.589022 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.693703 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:54.701133 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.703335 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:54.704076 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.711754 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7634984e-e760-4800-8b7e-c740ed5db3a0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:54.782550 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.801989 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Transition to ACTIVE for port object 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:54.814519 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:54.814957 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.826519 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Pushing event updated for resources: {'Network': ['ID=f1496a40-5ccd-4222-984b-dbffd9fdd474,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:54.830212 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:54.850309 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network f1496a40-5ccd-4222-984b-dbffd9fdd474 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 11:59:54.866846 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:54.871788 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:54.891171 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.7002707 May 31 11:59:54.895587 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36972) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:54.901144 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Request body: {'subnet': {'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-1085702316-subnet', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:54.906490 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:54.910345 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Usage tracker for resource:subnet and tenant:00afe2f4c68c431385f2a4b6061b0f72 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:54.913315 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:54.919936 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:55.038256 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Made reservation on behalf of 00afe2f4c68c431385f2a4b6061b0f72 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 11:59:55.044658 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:55.070796 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-09bb8d30-7092-4d21-b8d5-908c3dcdf643 None None] Pushing event updated for resources: {'Port': ['ID=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:55.070796 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-09bb8d30-7092-4d21-b8d5-908c3dcdf643 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:55.152411 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Attempting to bind port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:59:55.158615 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Attempting to bind port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:59:55.159994 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Attempting to bind port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 on network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:59:55.167162 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 59, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:59:55.167844 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Checking segment: {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:55.170092 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:55.170498 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Bound using segment: {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:59:55.170993 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Bound port: 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:59:55.171418 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:55.267530 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] For port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:59:55.270624 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] For port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:59:55.353502 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:55.360502 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-1c729d60-2118-42fd-af7d-7703356389a4 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:55.442283 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:55.442283 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:55.515830 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-09bb8d30-7092-4d21-b8d5-908c3dcdf643 None None] Pushing event updated for resources: {'Port': ['ID=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,revision_number=4']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:55.555917 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-09bb8d30-7092-4d21-b8d5-908c3dcdf643 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:55.662801 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:55.667714 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:55.690788 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:55.737663 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Pushing event updated for resources: {'Subnet': ['ID=2cdd9ddd-2036-4d15-a3ee-da37da24fae1,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:55.742849 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:55.755647 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Checking segment: {'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:55.756028 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Network hostable DHCP agents. Network: f1496a40-5ccd-4222-984b-dbffd9fdd474, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 11:59:55.855227 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Network f1496a40-5ccd-4222-984b-dbffd9fdd474 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 11:59:55.946440 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, f1496a40-5ccd-4222-984b-dbffd9fdd474) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:55.955840 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e72cb50c-18ed-47a3-9682-9cc36a2e6632 None None] Network f1496a40-5ccd-4222-984b-dbffd9fdd474 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:55.957487 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 11:59:55.958680 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.051414 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:55Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:56.055557 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c3866856-94b0-4daa-b8f9-d8256184d7b6 None None] The sync data for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi is: [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:55Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 11:59:56.066949 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-5ddd3a16-7c9d-4903-bb84-68664eae8547 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.1700158 May 31 11:59:56.069504 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 36994) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:56.074928 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 11:59:56.081032 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:56.085148 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Usage tracker for resource:router and tenant:00afe2f4c68c431385f2a4b6061b0f72 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:56.088223 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:56.095227 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:56.198381 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.221883 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.222942 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-f1496a40-5ccd-4222-984b-dbffd9fdd474', 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'fixed_ips': [{'subnet_id': '2cdd9ddd-2036-4d15-a3ee-da37da24fae1'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 11:59:56.223767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.223904 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.248939 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.393231 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Allocated IP 172.20.0.2 (f1496a40-5ccd-4222-984b-dbffd9fdd474/2cdd9ddd-2036-4d15-a3ee-da37da24fae1/6c20d80c-0f24-4be2-bace-18598335a7ae) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:56.631134 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.659606 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Transition to ACTIVE for port object 6c20d80c-0f24-4be2-bace-18598335a7ae will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:56.679438 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.688018 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Transition to ACTIVE for port object 6c20d80c-0f24-4be2-bace-18598335a7ae will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 11:59:56.740239 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:56.851784 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.860390 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:56.899518 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:56.923386 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:56.923763 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:56.989768 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Network f1496a40-5ccd-4222-984b-dbffd9fdd474 is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:56.995379 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:57.011462 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:57.013334 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Pushing event updated for resources: {'Port': ['ID=6c20d80c-0f24-4be2-bace-18598335a7ae,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:57.018806 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:57.157916 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Attempting to bind port 6c20d80c-0f24-4be2-bace-18598335a7ae on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 11:59:57.158575 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Attempting to bind port 6c20d80c-0f24-4be2-bace-18598335a7ae by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 11:59:57.159134 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Attempting to bind port 6c20d80c-0f24-4be2-bace-18598335a7ae on network f1496a40-5ccd-4222-984b-dbffd9fdd474 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 11:59:57.170004 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 59, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 11:59:57.170381 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Checking segment: {'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 11:59:57.170787 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.171183 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Bound using segment: {'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 11:59:57.171698 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Bound port: 6c20d80c-0f24-4be2-bace-18598335a7ae, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 11:59:57.172075 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.186831 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.263627 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:57.297946 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] For port 6c20d80c-0f24-4be2-bace-18598335a7ae, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 11:59:57.301371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] For port 6c20d80c-0f24-4be2-bace-18598335a7ae, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=6c20d80c-0f24-4be2-bace-18598335a7ae,segment=NetworkSegment(29648ab6-8c7b-47d7-9f64-2b32a50ee3f7),segment_id=29648ab6-8c7b-47d7-9f64-2b32a50ee3f7)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 11:59:57.377746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.381351 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.383273 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:57.419502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.426962 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.472445 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:57.486413 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:57.486911 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.052s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:57.569685 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Pushing event updated for resources: {'Port': ['ID=6c20d80c-0f24-4be2-bace-18598335a7ae,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:57.579593 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:57.609933 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Allocated IP 172.27.1.134 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/35f9ff48-03b5-48c7-8a4e-da38e484d698) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 11:59:57.816714 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.958064 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:57.963352 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-338f22e5-5038-4047-b838-7c739bc0545f tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:58.040415 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 11:59:58.043468 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 11:59:58.047441 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 11:59:58.081308 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Pushing event updated for resources: {'Port': ['ID=35f9ff48-03b5-48c7-8a4e-da38e484d698,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:58.085593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:58.488336 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-7a6f1732-d17a-4fb9-bab5-febda8b429ca None None] Network f1496a40-5ccd-4222-984b-dbffd9fdd474 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 11:59:58.594412 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:58.714404 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-6ba681c1-3391-4c3e-9ce6-5d957a04fd56 None None] Provisioning for port 6c20d80c-0f24-4be2-bace-18598335a7ae completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:58.794474 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 6c20d80c-0f24-4be2-bace-18598335a7ae up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:58.930650 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 6c20d80c-0f24-4be2-bace-18598335a7ae completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 11:59:58.938912 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 6c20d80c-0f24-4be2-bace-18598335a7ae triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:58.942919 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:58.968206 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.059994 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-fb0e8189-87ad-4417-a7c9-4a985e47aab2 None None] Provisioning complete for port 6c20d80c-0f24-4be2-bace-18598335a7ae triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:59.060521 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-fb0e8189-87ad-4417-a7c9-4a985e47aab2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.237428 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.267572 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-fb0e8189-87ad-4417-a7c9-4a985e47aab2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.299037 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 6c20d80c-0f24-4be2-bace-18598335a7ae, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=6c20d80c-0f24-4be2-bace-18598335a7ae,segment=NetworkSegment(29648ab6-8c7b-47d7-9f64-2b32a50ee3f7),segment_id=29648ab6-8c7b-47d7-9f64-2b32a50ee3f7)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 11:59:59.387399 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.388381 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 11:59:59.416228 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Router e1489825-752a-4858-b711-8618e0a7bb8a is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99039) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 11:59:59.467425 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:59.582529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37042) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:59.605839 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.db.api [None req-fb0e8189-87ad-4417-a7c9-4a985e47aab2 None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99041) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 31 11:59:59.606438 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_db.api [None req-fb0e8189-87ad-4417-a7c9-4a985e47aab2 None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 31 11:59:59.607217 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=6c20d80c-0f24-4be2-bace-18598335a7ae,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 11:59:59.610963 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 11:59:59.611338 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.639380 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.172s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 11:59:59.640482 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 11:59:59.654588 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Sync routers for ids ['e1489825-752a-4858-b711-8618e0a7bb8a'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 11:59:59.674371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Routers {'e1489825-752a-4858-b711-8618e0a7bb8a'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 11:59:59.675002 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['e1489825-752a-4858-b711-8618e0a7bb8a'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 11:59:59.715760 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.731182 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 11:59:59.735846 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Usage tracker for resource:security_group_rule and tenant:1a7a0b775fcf41239f1812846351d05c is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 11:59:59.738831 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 11:59:59.744887 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 11:59:59.752279 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.754593 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f7310fd1-401d-4083-a172-f6ae7aef1c5e admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 921 time: 3.6840396 May 31 11:59:59.758723 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37048) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:59.862572 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.913667 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Set router e1489825-752a-4858-b711-8618e0a7bb8a gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 11:59:59.935067 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 11:59:59.944364 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3b58b77d-6f92-4e49-a7fc-471bfcfc25c4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 11:59:59.984465 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=1a7a0b775fcf41239f1812846351d05c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3996396 May 31 11:59:59.987018 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37054) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 11:59:59.989799 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f25fed42-25c9-46f8-8d19-69ccec600da7,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:00.002301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a3fb6079-05d8-433d-b021-621b22bc541b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:00.057785 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a06d6205-ae67-4b46-89e1-5218054214aa admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.089778 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a06d6205-ae67-4b46-89e1-5218054214aa admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.268816 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T11:59:58Z', 'revision_number': 3, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T11:59:57Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:00.273043 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Checking router: e1489825-752a-4858-b711-8618e0a7bb8a for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:00:00.273415 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Updating router e1489825-752a-4858-b711-8618e0a7bb8a port 35f9ff48-03b5-48c7-8a4e-da38e484d698 binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 12:00:00.331786 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.381317 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a06d6205-ae67-4b46-89e1-5218054214aa admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.386634 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-23785343-a460-450b-97b3-d9c507f28424 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:00.399993 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a06d6205-ae67-4b46-89e1-5218054214aa admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f25fed42-25c9-46f8-8d19-69ccec600da7,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:00.402875 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a06d6205-ae67-4b46-89e1-5218054214aa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:00.404026 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a06d6205-ae67-4b46-89e1-5218054214aa admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.404968 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a06d6205-ae67-4b46-89e1-5218054214aa admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/f25fed42-25c9-46f8-8d19-69ccec600da7 HTTP/1.1" status: 204 len: 168 time: 0.4169080 May 31 12:00:00.526183 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.744598 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 6c20d80c-0f24-4be2-bace-18598335a7ae up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:00:00.748446 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Allocated IP 172.20.0.1 (f1496a40-5ccd-4222-984b-dbffd9fdd474/2cdd9ddd-2036-4d15-a3ee-da37da24fae1/e3e2dad9-58b2-48c0-8b94-ffde87a6b32a) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:00:00.817798 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.834465 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Transition to ACTIVE for port object 35f9ff48-03b5-48c7-8a4e-da38e484d698 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:00:00.846266 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 6c20d80c-0f24-4be2-bace-18598335a7ae triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:00.847096 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.905087 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.914650 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:00.922010 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:00.926818 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Transition to ACTIVE for port object e3e2dad9-58b2-48c0-8b94-ffde87a6b32a will not be triggered until provisioned by entity DHCP. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:00:01.003635 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Attempting to bind port 35f9ff48-03b5-48c7-8a4e-da38e484d698 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:00:01.003997 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Attempting to bind port 35f9ff48-03b5-48c7-8a4e-da38e484d698 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:00:01.004372 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Attempting to bind port 35f9ff48-03b5-48c7-8a4e-da38e484d698 on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:00:01.015191 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c69390d6-5260-479c-8eda-b9f4b35af6a5 None None] Pushing event updated for resources: {'Port': ['ID=35f9ff48-03b5-48c7-8a4e-da38e484d698,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:01.018446 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 59, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:00:01.018825 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:00:01.019451 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.019663 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:00:01.020145 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Bound port: 35f9ff48-03b5-48c7-8a4e-da38e484d698, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:00:01.020515 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.049085 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-c69390d6-5260-479c-8eda-b9f4b35af6a5 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.128s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:01.056413 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.065478 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-29ea84e1-ca2e-4c6c-ac40-4cf33c840d19 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:01.148098 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] For port 35f9ff48-03b5-48c7-8a4e-da38e484d698, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:00:01.150007 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Network f1496a40-5ccd-4222-984b-dbffd9fdd474 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:00:01.151286 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] For port 35f9ff48-03b5-48c7-8a4e-da38e484d698, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=35f9ff48-03b5-48c7-8a4e-da38e484d698,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:00:01.153839 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:00:01.157817 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:00:01.218398 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Pushing event updated for resources: {'Port': ['ID=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:01.224199 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:01.234576 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.295870 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.304876 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:01.400011 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Routers {'e1489825-752a-4858-b711-8618e0a7bb8a'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:00:01.400637 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['e1489825-752a-4858-b711-8618e0a7bb8a'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:01.481235 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c69390d6-5260-479c-8eda-b9f4b35af6a5 None None] Pushing event updated for resources: {'Port': ['ID=35f9ff48-03b5-48c7-8a4e-da38e484d698,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:01.494337 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-c69390d6-5260-479c-8eda-b9f4b35af6a5 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:01.575898 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-26e21309-5a72-40ce-a4c3-59e1e238ba71 None None] Provisioning for port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:00:01.584978 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-26e21309-5a72-40ce-a4c3-59e1e238ba71 None None] Provisioning complete for port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:01.585439 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-26e21309-5a72-40ce-a4c3-59e1e238ba71 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.595461 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.663478 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] Set router e1489825-752a-4858-b711-8618e0a7bb8a gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:00:01.707274 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-26e21309-5a72-40ce-a4c3-59e1e238ba71 None None] Port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:00:01.889285 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.969901 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:01.997118 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-439d4385-a7d3-4cf7-95a5-8437637439e8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:02.003507 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T11:59:58Z', 'revision_number': 3, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T12:00:01Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:02.008148 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dfe5f060-867b-4b86-857b-2669a36f4c33 None None] The sync data for ids ['e1489825-752a-4858-b711-8618e0a7bb8a'] in dv-cinder-791239-4-iscsi is: [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T11:59:58Z', 'revision_number': 3, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T12:00:01Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:00:02.087104 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:02.091056 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Pushing event updated for resources: {'Port': ['ID=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:02.095676 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:02.267433 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Network f1496a40-5ccd-4222-984b-dbffd9fdd474 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:00:02.268596 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:00:02.491554 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Router e1489825-752a-4858-b711-8618e0a7bb8a has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:00:02.572326 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-0a28c208-e6ac-4502-8c68-67188a4659bc None None] Provisioning complete for port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:02.572707 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-0a28c208-e6ac-4502-8c68-67188a4659bc None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:02.659177 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-0a28c208-e6ac-4502-8c68-67188a4659bc None None] Port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:00:02.711149 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:00:02.735737 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-327a7cc7-2cf0-4061-a24e-24edc35dff76 admin admin] 10.222.0.48 "PUT /v2.0/routers/e1489825-752a-4858-b711-8618e0a7bb8a/add_router_interface HTTP/1.1" status: 200 len: 503 time: 2.9774499 May 31 12:00:02.761310 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 35f9ff48-03b5-48c7-8a4e-da38e484d698 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:00:02.903726 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 35f9ff48-03b5-48c7-8a4e-da38e484d698 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:00:02.916584 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 35f9ff48-03b5-48c7-8a4e-da38e484d698 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:02.916976 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:03.112251 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:03.131137 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 35f9ff48-03b5-48c7-8a4e-da38e484d698, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=35f9ff48-03b5-48c7-8a4e-da38e484d698,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:00:03.180686 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:03.182660 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-3b981d79-af07-4458-a045-674e51c68c3b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:03.253522 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=35f9ff48-03b5-48c7-8a4e-da38e484d698,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:03.256476 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:03.554667 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37122) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:03.560620 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-1046337977', 'description': 'tempest-secgroup-smoke-1046337977 description', 'project_id': 'bbcb901221a14602a1f2f8b198287636'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:03.567112 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:03.570932 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:security_group and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:03.573871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:03.580892 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:03.696824 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:03.719105 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:03.723132 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:security_group_rule and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:03.723673 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Sync routers for ids ['e1489825-752a-4858-b711-8618e0a7bb8a'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:00:03.726237 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:03.732097 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:03.734214 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Routers {'e1489825-752a-4858-b711-8618e0a7bb8a'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:00:03.734986 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['e1489825-752a-4858-b711-8618e0a7bb8a'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:03.808911 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:03.901099 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:03.908171 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-70f55d7c-fe8e-4243-ac10-5f86bfcffd2c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:03.930994 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Set router e1489825-752a-4858-b711-8618e0a7bb8a gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:00:03.950065 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'SecurityGroup': ['ID=3bc79f51-0858-405c-97be-c79308faa377,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:03.952846 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:04.054307 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.055452 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1794 time: 0.4996071 May 31 12:00:04.058359 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37130) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:04.072208 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'security_group_rule': {'security_group_id': '3bc79f51-0858-405c-97be-c79308faa377', 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:04.072208 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:04.074121 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:security_group_rule and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:04.077133 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:04.083827 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:04.154076 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.235127 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.287084 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T12:00:01Z', 'revision_number': 4, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T12:00:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'e3e2dad9-58b2-48c0-8b94-ffde87a6b32a', 'name': '', 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'mac_address': 'fa:16:3e:89:25:0a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_interface', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '2cdd9ddd-2036-4d15-a3ee-da37da24fae1', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:00:00Z', 'updated_at': '2021-05-31T12:00:01Z', 'revision_number': 2, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'subnets': [{'id': '2cdd9ddd-2036-4d15-a3ee-da37da24fae1', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:04.290845 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Checking router: e1489825-752a-4858-b711-8618e0a7bb8a for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:00:04.291198 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Updating router e1489825-752a-4858-b711-8618e0a7bb8a port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 12:00:04.306740 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.315985 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:04.336994 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=c887eb7f-00ff-4018-b1be-ee921df4ffcd,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:04.341099 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:04.343976 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.426615 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.430484 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.3686342 May 31 12:00:04.430805 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37138) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:04.437298 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'security_group_rule': {'security_group_id': '3bc79f51-0858-405c-97be-c79308faa377', 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:04.442585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:04.446399 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:security_group_rule and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:04.449243 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:04.455978 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:04.521369 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.589304 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.643996 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.646986 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-fd54047c-31c6-4d80-8cd8-268ef43d7419 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:04.677236 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=41501f85-e99d-40f8-ab00-56ed959c8c37,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:04.684693 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:04.704782 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.724009 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 35f9ff48-03b5-48c7-8a4e-da38e484d698 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:00:04.731131 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Transition to ACTIVE for port object e3e2dad9-58b2-48c0-8b94-ffde87a6b32a will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:00:04.758516 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.759729 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-2e286782-19df-4ed7-8f4d-0fdc656af46f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.3279767 May 31 12:00:04.762451 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37148) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:04.768492 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'security_group_rule': {'security_group_id': '3bc79f51-0858-405c-97be-c79308faa377', 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:04.774726 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:04.778717 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:security_group_rule and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:04.781813 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:04.788128 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:04.815069 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 35f9ff48-03b5-48c7-8a4e-da38e484d698 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:04.815449 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.827228 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.831270 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-297bb2d7-0ad5-499d-86e0-0b702be878b3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:04.857664 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:04.896950 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9f112a3-346c-4109-8cc3-26bc66aed0ad None None] Pushing event updated for resources: {'Port': ['ID=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:04.910342 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-c9f112a3-346c-4109-8cc3-26bc66aed0ad None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:04.946534 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.044058 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.053665 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e21c525b-9b4d-40c2-a114-3f67a95897c7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:05.084918 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=3d328fc7-a32c-4c6b-bb47-afe435b6a5ec,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:05.087654 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:05.088768 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Attempting to bind port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:00:05.089223 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Attempting to bind port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:00:05.089737 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Attempting to bind port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a on network f1496a40-5ccd-4222-984b-dbffd9fdd474 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:00:05.096664 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 11, 59, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:00:05.099846 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Checking segment: {'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:00:05.099846 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.099846 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Bound using segment: {'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:00:05.099846 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Bound port: e3e2dad9-58b2-48c0-8b94-ffde87a6b32a, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 365, 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:00:05.099846 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.188436 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] For port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:00:05.191991 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] For port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,segment=NetworkSegment(29648ab6-8c7b-47d7-9f64-2b32a50ee3f7),segment_id=29648ab6-8c7b-47d7-9f64-2b32a50ee3f7)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:00:05.193863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.195103 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-66147832-5778-4d27-900a-c9c02d1bbaf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.4317389 May 31 12:00:05.198011 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37156) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:05.204183 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'security_group_rule': {'security_group_id': '3bc79f51-0858-405c-97be-c79308faa377', 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:05.210185 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:05.213968 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:security_group_rule and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:05.216960 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:05.223701 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:05.329864 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.335003 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-62061ac9-e43f-4917-963f-82f698d8fa30 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:05.365250 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.407846 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Routers {'e1489825-752a-4858-b711-8618e0a7bb8a'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:00:05.408463 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T11:59:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['e1489825-752a-4858-b711-8618e0a7bb8a'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:05.440959 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.490527 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9f112a3-346c-4109-8cc3-26bc66aed0ad None None] Pushing event updated for resources: {'Port': ['ID=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,revision_number=4']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:05.496824 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-c9f112a3-346c-4109-8cc3-26bc66aed0ad None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:05.524008 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.526069 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:05.563528 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=7f4528fd-e69f-4ebf-8a56-d102fb7cd284,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:05.566470 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:05.632767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.634306 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-551a8e4c-1ee6-4f43-9d4b-6c7cad50021c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.4352369 May 31 12:00:05.637006 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37164) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:05.639100 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] Set router e1489825-752a-4858-b711-8618e0a7bb8a gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:00:05.643950 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'security_group_rule': {'security_group_id': '3bc79f51-0858-405c-97be-c79308faa377', 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:05.651505 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:05.658323 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:security_group_rule and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:05.663115 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:05.669254 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:05.786669 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.securitygroups_db [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Project bbcb901221a14602a1f2f8b198287636 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 31 12:00:05.794445 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.857817 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.923956 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:05.925812 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-dbbcc826-9ee2-4096-a734-04ea39dc7598 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:05.947209 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=9a2f4056-4e52-479e-a727-48e9e4d5a36c,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:05.951201 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:05.963301 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T12:00:01Z', 'revision_number': 4, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T12:00:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'e3e2dad9-58b2-48c0-8b94-ffde87a6b32a', 'name': '', 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'mac_address': 'fa:16:3e:89:25:0a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_interface', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '2cdd9ddd-2036-4d15-a3ee-da37da24fae1', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:00:00Z', 'updated_at': '2021-05-31T12:00:05Z', 'revision_number': 4, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'subnets': [{'id': '2cdd9ddd-2036-4d15-a3ee-da37da24fae1', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:05.967237 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-16732a6d-ae06-4788-bdc6-bb017dcc5c2b None None] The sync data for ids ['e1489825-752a-4858-b711-8618e0a7bb8a'] in dv-cinder-791239-4-iscsi is: [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T12:00:01Z', 'revision_number': 4, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T12:00:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'e3e2dad9-58b2-48c0-8b94-ffde87a6b32a', 'name': '', 'network_id': 'f1496a40-5ccd-4222-984b-dbffd9fdd474', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'mac_address': 'fa:16:3e:89:25:0a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_interface', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '2cdd9ddd-2036-4d15-a3ee-da37da24fae1', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:00:00Z', 'updated_at': '2021-05-31T12:00:05Z', 'revision_number': 4, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'subnets': [{'id': '2cdd9ddd-2036-4d15-a3ee-da37da24fae1', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:00:06.060620 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:06.062430 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c0609163-4c0c-4b94-b7a6-6eb54c99bc6c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.4235573 May 31 12:00:06.065174 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37172) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:06.071132 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'security_group_rule': {'security_group_id': '3bc79f51-0858-405c-97be-c79308faa377', 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:06.077405 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:06.081603 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:security_group_rule and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:06.084771 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:06.090782 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:06.186394 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.securitygroups_db [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Project bbcb901221a14602a1f2f8b198287636 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 31 12:00:06.193298 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:06.262716 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:06.352125 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:06.358227 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:06.378345 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=cbbaa2dd-f1a8-400c-a085-4841c9c1b9e1,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:06.380856 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:06.455049 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:06.456274 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-11f39cd0-437f-4440-ae1f-7784a3ba089c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.3900402 May 31 12:00:06.763288 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device e3e2dad9-58b2-48c0-8b94-ffde87a6b32a up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:00:06.952211 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:00:06.957544 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:06.958160 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:07.076949 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:07.109164 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,segment=NetworkSegment(29648ab6-8c7b-47d7-9f64-2b32a50ee3f7),segment_id=29648ab6-8c7b-47d7-9f64-2b32a50ee3f7)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:00:07.221503 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:07.222623 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:07.294447 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:07.298020 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:08.703693 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device e3e2dad9-58b2-48c0-8b94-ffde87a6b32a up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:00:08.770383 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:08.770765 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:09.055363 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:12.468814 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:00:14.194418 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37322) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:14.258019 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] Deleting subnet ad6c0027-9f7a-4d64-a9a0-1449192c5ee4 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:00:14.261005 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:14.322156 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:14.378656 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:14.803931 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:14.808852 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:14.843825 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:14.847609 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] Pushing event deleted for resources: {'Subnet': ['ID=ad6c0027-9f7a-4d64-a9a0-1449192c5ee4,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:14.849781 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:14.852871 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-b7650e6e-80c1-4100-ba70-66973f2c12fc admin admin] 10.222.0.48 "DELETE /v2.0/subnets/ad6c0027-9f7a-4d64-a9a0-1449192c5ee4 HTTP/1.1" status: 204 len: 168 time: 0.6574500 May 31 12:00:14.855361 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37330) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:14.965130 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.090474 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.108281 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['59773d4e-a35c-4ba4-8bef-ff4d4070804c']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:15.121919 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'ce07ff9d-e078-4661-b74a-f01d070a87f3') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:15.122218 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'ce07ff9d-e078-4661-b74a-f01d070a87f3') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:15.134310 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.150800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.157712 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Releasing vlan 375 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:00:15.158176 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:00:15.171181 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:00:15.329867 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.334657 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:15.351919 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.358650 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Pushing event deleted for resources: {'Network': ['ID=59773d4e-a35c-4ba4-8bef-ff4d4070804c,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:15.360834 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:15.362331 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a9cddde6-7a67-457a-ba38-466857322e60 admin admin] 10.222.0.48 "DELETE /v2.0/networks/59773d4e-a35c-4ba4-8bef-ff4d4070804c HTTP/1.1" status: 204 len: 168 time: 0.5060761 May 31 12:00:15.365086 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37342) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:15.423723 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] Deleting subnet 3232ed29-0c0e-4db0-a48b-7e3bd671f4ff {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:00:15.426778 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.480035 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.536086 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.878298 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.882358 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1dc98d60-8dab-4082-8cb0-03e3c68c290b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:15.916011 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=3232ed29-0c0e-4db0-a48b-7e3bd671f4ff,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:15.919034 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:15.920656 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:15.926283 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a3618db8-17bc-4cc4-8fba-a4117847ccd5 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/3232ed29-0c0e-4db0-a48b-7e3bd671f4ff HTTP/1.1" status: 204 len: 168 time: 0.5602403 May 31 12:00:15.928691 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37346) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:16.038465 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:16.164272 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:16.177222 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['49520081-8b4f-454e-b744-1f256a1202d2']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:16.188966 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '59945c0f-16e2-4afd-b32b-ccf004cf3f18') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:16.189258 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '59945c0f-16e2-4afd-b32b-ccf004cf3f18') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:16.200829 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:16.216688 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:16.223288 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Releasing vlan 369 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:00:16.223774 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:00:16.237873 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:00:16.426180 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:16.429703 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-af307107-9afc-41ad-bdad-098681b7c8c5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:16.442457 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:16.451104 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Pushing event deleted for resources: {'Network': ['ID=49520081-8b4f-454e-b744-1f256a1202d2,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:16.454538 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:16.456911 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f443c922-c835-4b7f-bad9-4570f693c605 admin admin] 10.222.0.48 "DELETE /v2.0/networks/49520081-8b4f-454e-b744-1f256a1202d2 HTTP/1.1" status: 204 len: 168 time: 0.5266409 May 31 12:00:16.837092 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37356) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:16.885009 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8290dce8-106f-4381-955e-ae62efda4500 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=1225cc76a5a0424ab646043550b5a0d8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0465877 May 31 12:00:16.887261 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37360) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:16.951231 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a60dcce3-cfb7-4bf3-8c4e-c44476a7cc88 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:17.067414 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a60dcce3-cfb7-4bf3-8c4e-c44476a7cc88 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:17.180186 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a60dcce3-cfb7-4bf3-8c4e-c44476a7cc88 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:17.183965 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0ba7fe90-41d6-4e65-852e-a6e5e1ce4d84 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:17.195147 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a60dcce3-cfb7-4bf3-8c4e-c44476a7cc88 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=56fa3b08-f1c3-4248-9175-95d2ebb4e120,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:17.197620 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a60dcce3-cfb7-4bf3-8c4e-c44476a7cc88 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:17.203517 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a60dcce3-cfb7-4bf3-8c4e-c44476a7cc88 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:17.204621 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a60dcce3-cfb7-4bf3-8c4e-c44476a7cc88 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/56fa3b08-f1c3-4248-9175-95d2ebb4e120 HTTP/1.1" status: 204 len: 168 time: 0.3164711 May 31 12:00:17.885832 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37372) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:17.935714 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-013c7c1a-bec7-4dfa-97ff-7ea7910d910a admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=e080a39f16314f14b44474063dd46cff&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0515933 May 31 12:00:17.938669 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37376) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:18.003202 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-82c2fae6-f36d-4c38-ba20-9db1bfa61312 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:18.028696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-82c2fae6-f36d-4c38-ba20-9db1bfa61312 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:18.214704 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-82c2fae6-f36d-4c38-ba20-9db1bfa61312 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:18.218441 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-13ab40c3-2733-480c-b7c7-ce1596fe57a6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:18.229250 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-82c2fae6-f36d-4c38-ba20-9db1bfa61312 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=25fb30a6-1542-40de-9b70-1bb2e82cb2c7,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:18.233071 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-82c2fae6-f36d-4c38-ba20-9db1bfa61312 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:18.236897 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-82c2fae6-f36d-4c38-ba20-9db1bfa61312 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:18.240265 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-82c2fae6-f36d-4c38-ba20-9db1bfa61312 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/25fb30a6-1542-40de-9b70-1bb2e82cb2c7 HTTP/1.1" status: 204 len: 168 time: 0.2984424 May 31 12:00:20.215294 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37472) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:20.334369 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:20.349355 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:20.353219 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Usage tracker for resource:security_group_rule and tenant:b8489621c31f4f8ebdb6c124a25f8d65 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:20.356073 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:20.362650 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:20.449852 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:20.551956 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:20.559337 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e19f00d6-abd5-4e1e-8747-cbfff7d86d89 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:20.601060 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=b8489621c31f4f8ebdb6c124a25f8d65&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3846948 May 31 12:00:20.603810 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e6ab00cc-857b-4d53-9008-77d8348b24a2,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:20.604569 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37484) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:20.606778 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:20.668991 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1fdffeba-5622-47f0-9131-ce529f6a66f2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:20.695037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1fdffeba-5622-47f0-9131-ce529f6a66f2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:20.865973 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1fdffeba-5622-47f0-9131-ce529f6a66f2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:20.870744 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0e2f1678-0fbd-442f-b5aa-4bcb946427f6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:20.882394 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1fdffeba-5622-47f0-9131-ce529f6a66f2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e6ab00cc-857b-4d53-9008-77d8348b24a2,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:20.885219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1fdffeba-5622-47f0-9131-ce529f6a66f2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:20.887217 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1fdffeba-5622-47f0-9131-ce529f6a66f2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:20.888188 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-1fdffeba-5622-47f0-9131-ce529f6a66f2 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/e6ab00cc-857b-4d53-9008-77d8348b24a2 HTTP/1.1" status: 204 len: 168 time: 0.2826536 May 31 12:00:20.902332 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ae728089-8962-4e1a-9b6c-91b610783ecb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:23.334403 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:23.476859 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-03e905e8-b163-4822-9a66-955cb5e3d80f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:24.648516 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:24.734805 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37536) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:24.853715 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:24.867352 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:24.871120 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Usage tracker for resource:security_group_rule and tenant:f5c7fac5fd95472fbfab34e7d80f76cb is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:24.873848 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:24.879520 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:24.959341 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:25.019861 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:25.034479 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b71ae402-2c4f-403d-bbda-511a2ff69adf admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:25.082848 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=f5c7fac5fd95472fbfab34e7d80f76cb&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3468876 May 31 12:00:25.083876 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7b14dae6-26fb-4b91-9bcd-595e97760b52,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:25.085646 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37544) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:25.092856 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.093313 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.094460 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.094866 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.095215 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.095561 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.095904 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.096321 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.096768 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.097114 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.097452 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.097879 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:25.103351 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-232776e9-1e91-48a2-b72d-860f7a610fea admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:25.145087 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8003c634-cb7b-4436-8cdc-2a784b9df50a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:25.173397 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8003c634-cb7b-4436-8cdc-2a784b9df50a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:25.311484 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8003c634-cb7b-4436-8cdc-2a784b9df50a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:25.317072 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-26687271-ded4-48cd-bbdf-4dc7fa22e023 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:25.332447 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8003c634-cb7b-4436-8cdc-2a784b9df50a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7b14dae6-26fb-4b91-9bcd-595e97760b52,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:25.335525 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8003c634-cb7b-4436-8cdc-2a784b9df50a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:25.339804 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8003c634-cb7b-4436-8cdc-2a784b9df50a admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:25.341305 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8003c634-cb7b-4436-8cdc-2a784b9df50a admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/7b14dae6-26fb-4b91-9bcd-595e97760b52 HTTP/1.1" status: 204 len: 168 time: 0.2543573 May 31 12:00:26.105333 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:26.382563 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-99e42a00-fb2a-45db-96f1-35157005d907 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Synchronizing usage tracker for tenant:bbcb901221a14602a1f2f8b198287636 on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:00:26.471170 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-99e42a00-fb2a-45db-96f1-35157005d907 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Unset dirty status for tenant:bbcb901221a14602a1f2f8b198287636 on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:00:26.472279 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-99e42a00-fb2a-45db-96f1-35157005d907 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-1046337977&fields=id&tenant_id=bbcb901221a14602a1f2f8b198287636 HTTP/1.1" status: 200 len: 267 time: 0.1312962 May 31 12:00:26.621025 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-cb1e6502-31ad-482c-8a7f-ed4fa3fd253d tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:26.621629 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-cb1e6502-31ad-482c-8a7f-ed4fa3fd253d tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Synchronizing usage tracker for tenant:bbcb901221a14602a1f2f8b198287636 on resource:network {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:00:26.682444 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-cb1e6502-31ad-482c-8a7f-ed4fa3fd253d tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Unset dirty status for tenant:bbcb901221a14602a1f2f8b198287636 on resource:network {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:00:26.683689 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-cb1e6502-31ad-482c-8a7f-ed4fa3fd253d tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/networks?id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 200 len: 809 time: 0.2047696 May 31 12:00:26.698520 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-0434e090-915f-431f-bac3-8eab8299d9bf tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/quotas/bbcb901221a14602a1f2f8b198287636 HTTP/1.1" status: 200 len: 375 time: 0.0093627 May 31 12:00:26.781831 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-8121d0c2-a6f6-446e-8c16-06471a58992a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0768726 May 31 12:00:26.916542 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-a2abc578-f4e4-4dd7-b41c-9f356ebbf36c service neutron] 10.222.0.48 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1232643 May 31 12:00:27.048769 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-d98247e0-82c8-42d6-84a0-e71c4fd2e3ea service neutron] 10.222.0.48 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1249113 May 31 12:00:27.966707 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f50a028-45e3-402e-b884-0cea1129c69c req-3b133270-c235-4212-a327-c4a4ff5bc13d tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 210 time: 0.0502174 May 31 12:00:29.121893 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-03e00a06-0b32-40de-ae90-14d5905421b2 req-70b04bfa-26d6-40a4-8c45-f2b7aed9b4b8 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 210 time: 0.0411453 May 31 12:00:30.111824 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-163f31c8-5e96-4a84-bafa-65c17d068972 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:30.113279 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-163f31c8-5e96-4a84-bafa-65c17d068972 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.47 "GET /v2.0/networks?id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 200 len: 809 time: 0.1470335 May 31 12:00:30.169889 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-915976bc-47a4-44eb-997c-1990a8cb837d tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.47 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=bbcb901221a14602a1f2f8b198287636 HTTP/1.1" status: 200 len: 380 time: 0.0459213 May 31 12:00:30.179573 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'port': {'device_id': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'admin_state_up': True, 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'security_groups': ['3bc79f51-0858-405c-97be-c79308faa377']}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:30.315669 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b586b5b5-4109-478e-b836-842cbd6f6ec9 req-fda4f62c-89eb-4ba2-932b-5c4194681e80 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 210 time: 0.0623527 May 31 12:00:30.364331 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:30.368348 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:port and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:30.371288 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for port was recalculated. Used quota:2. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:30.377142 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:30.460037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Made reservation on behalf of bbcb901221a14602a1f2f8b198287636 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:00:30.463612 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:30.707598 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Allocated IP 172.20.0.6 (fc7e08b1-448f-4bc6-8c22-cdf76e730c2f/3a19e015-3084-4732-bb65-ea124df3118f/5add15c1-9f55-440c-b343-5d2f806a63c2) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:00:30.898694 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:30.917618 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Transition to ACTIVE for port object 5add15c1-9f55-440c-b343-5d2f806a63c2 will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:00:31.014586 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:31.022040 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1417b134-9b07-4934-bcb1-d59c7b765391 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:31.086441 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:00:31.089589 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:00:31.093454 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:00:31.144910 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:31.150466 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.128s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:31.176845 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:31.177524 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:31.250053 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.0736587 May 31 12:00:31.262871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Request body: {'port': {'device_id': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:31.391190 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:31.431729 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-ac028a6c-dc2a-4262-ade2-43460f43515d None None] Provisioning for port 5add15c1-9f55-440c-b343-5d2f806a63c2 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:00:31.438822 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-ac028a6c-dc2a-4262-ade2-43460f43515d None None] Provisioning complete for port 5add15c1-9f55-440c-b343-5d2f806a63c2 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:31.439716 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ac028a6c-dc2a-4262-ade2-43460f43515d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:31.507840 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-ac028a6c-dc2a-4262-ade2-43460f43515d None None] Port 5add15c1-9f55-440c-b343-5d2f806a63c2 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:00:31.675551 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e763066a-84b3-48df-940c-7bc7d010d74c req-ac046732-36f3-4527-aa33-f6f9b0dd8ac8 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:31.676258 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-e763066a-84b3-48df-940c-7bc7d010d74c req-ac046732-36f3-4527-aa33-f6f9b0dd8ac8 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Synchronizing usage tracker for tenant:bbcb901221a14602a1f2f8b198287636 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:00:31.825831 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 5add15c1-9f55-440c-b343-5d2f806a63c2 {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:00:31.834332 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:31.838419 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-e763066a-84b3-48df-940c-7bc7d010d74c req-ac046732-36f3-4527-aa33-f6f9b0dd8ac8 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Unset dirty status for tenant:bbcb901221a14602a1f2f8b198287636 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:00:31.839566 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e763066a-84b3-48df-940c-7bc7d010d74c req-ac046732-36f3-4527-aa33-f6f9b0dd8ac8 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 931 time: 0.3931258 May 31 12:00:31.864726 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Transition to ACTIVE for port object 5add15c1-9f55-440c-b343-5d2f806a63c2 will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:00:31.890027 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e763066a-84b3-48df-940c-7bc7d010d74c req-5fcef105-08a5-4cc1-bf52-9f1da10861e1 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0439727 May 31 12:00:31.940059 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:31.945068 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:31.977451 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e538a11e-9e67-40d3-953f-c72e6d772e04 req-89d2e7ab-0a61-4c75-9803-4842a3fe4eb1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:31.979064 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e538a11e-9e67-40d3-953f-c72e6d772e04 req-89d2e7ab-0a61-4c75-9803-4842a3fe4eb1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0846996 May 31 12:00:32.028498 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e538a11e-9e67-40d3-953f-c72e6d772e04 req-e62f0be2-9699-4fbd-8e7e-e9c8322ceb54 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0415611 May 31 12:00:32.107253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Pushing event updated for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:32.114358 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.169s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:32.189083 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Attempting to bind port 5add15c1-9f55-440c-b343-5d2f806a63c2 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:00:32.189787 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Attempting to bind port 5add15c1-9f55-440c-b343-5d2f806a63c2 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:00:32.189787 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Attempting to bind port 5add15c1-9f55-440c-b343-5d2f806a63c2 on network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:00:32.194412 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Port 5add15c1-9f55-440c-b343-5d2f806a63c2 on network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 12:00:32.194732 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Attempting to bind port 5add15c1-9f55-440c-b343-5d2f806a63c2 on network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:00:32.199917 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 0, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:00:32.204536 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Checking segment: {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:00:32.204536 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Bound using segment: {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:00:32.204536 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Bound port: 5add15c1-9f55-440c-b343-5d2f806a63c2, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'edd9824c-4e69-4a31-bd8c-e501618fb7bb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 397, 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:00:32.204536 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:32.289479 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:00:32.292417 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:00:32.375303 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:32.379453 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:32.438814 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:32.439357 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:32.440669 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] 10.222.0.47 "PUT /v2.0/ports/5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 200 len: 1093 time: 1.1821456 May 31 12:00:32.444546 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:00:32.446875 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-ca0450b3-297a-4c08-8786-4044e2e4250f" -d '{"events": [{"name": "network-changed", "server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:00:32.469611 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Pushing event updated for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:32.473086 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:32.492156 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:00:32 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-6109299d-41a0-4cb4-8f02-b5604dcb3a65 x-openstack-request-id: req-6109299d-41a0-4cb4-8f02-b5604dcb3a65 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:00:32.492483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:00:32.492803 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-6109299d-41a0-4cb4-8f02-b5604dcb3a65 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:00:32.493278 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'status': 'completed', 'code': 200} May 31 12:00:32.585683 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca0450b3-297a-4c08-8786-4044e2e4250f req-b119e0cb-3bd5-4d43-9da2-f9f692271bde service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:32.587170 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca0450b3-297a-4c08-8786-4044e2e4250f req-b119e0cb-3bd5-4d43-9da2-f9f692271bde service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1096 time: 0.0748978 May 31 12:00:32.729372 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca0450b3-297a-4c08-8786-4044e2e4250f req-dbfde1cc-ad67-4f0a-80c8-be5c229fc66a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:32.730765 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca0450b3-297a-4c08-8786-4044e2e4250f req-dbfde1cc-ad67-4f0a-80c8-be5c229fc66a service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=bbcb901221a14602a1f2f8b198287636&shared=False HTTP/1.1" status: 200 len: 910 time: 0.1358898 May 31 12:00:32.947262 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca0450b3-297a-4c08-8786-4044e2e4250f req-410eed5d-d1d7-4f4b-b9dd-8dae332e3699 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:32.948703 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca0450b3-297a-4c08-8786-4044e2e4250f req-410eed5d-d1d7-4f4b-b9dd-8dae332e3699 service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2113657 May 31 12:00:33.116613 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-7075ca2b-7737-4fbd-87f7-e0a003b3fd5a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:33.118152 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-7075ca2b-7737-4fbd-87f7-e0a003b3fd5a service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1096 time: 0.1310685 May 31 12:00:33.129517 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-99e08fe1-18f9-4578-bafa-0b4a40301b60 req-c8d6a2d3-3aae-4d1a-9276-5dce3f93e945 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:33.130910 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-99e08fe1-18f9-4578-bafa-0b4a40301b60 req-c8d6a2d3-3aae-4d1a-9276-5dce3f93e945 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.1061676 May 31 12:00:33.205903 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-fa742602-4232-4d3b-8945-48ce507ca71b service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 200 len: 217 time: 0.0739534 May 31 12:00:33.227891 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-938d8795-d3d7-4db5-bd07-55c300f43bf6 req-3767ea0f-2d7d-4acf-bc16-6ac2e34bcfe4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:33.229343 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-938d8795-d3d7-4db5-bd07-55c300f43bf6 req-3767ea0f-2d7d-4acf-bc16-6ac2e34bcfe4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0844104 May 31 12:00:33.274694 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-938d8795-d3d7-4db5-bd07-55c300f43bf6 req-cf8349ba-83b8-49f4-aabf-cd73a19d4956 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393586 May 31 12:00:33.276003 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-99e08fe1-18f9-4578-bafa-0b4a40301b60 req-93db8371-3123-4ac2-a565-a3d333011213 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0903356 May 31 12:00:33.294183 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-a6c79d10-a709-4ef2-93a1-9001d512b728 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:33.295773 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-a6c79d10-a709-4ef2-93a1-9001d512b728 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=3a19e015-3084-4732-bb65-ea124df3118f HTTP/1.1" status: 200 len: 848 time: 0.0827553 May 31 12:00:33.371005 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-fd615f07-f7ba-48a0-ab26-8c9d73bd23ad service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:33.372440 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-fd615f07-f7ba-48a0-ab26-8c9d73bd23ad service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0702507 May 31 12:00:33.517272 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-b5ab2ca2-dc78-4064-8718-e1448f1b5220 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1363561 May 31 12:00:33.649911 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bbbdcbf-8cbc-4011-890b-87a72cf65790 req-5dc9dcec-5c67-4eee-8eb5-2df59e38bc0a service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1264334 May 31 12:00:34.463930 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5f697ea5-2a49-4285-a414-18626190a8f6 req-77c87c22-ab8d-47ad-b273-6bc3fbd6ba78 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:34.464623 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-5f697ea5-2a49-4285-a414-18626190a8f6 req-77c87c22-ab8d-47ad-b273-6bc3fbd6ba78 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Synchronizing usage tracker for tenant:bbcb901221a14602a1f2f8b198287636 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:00:34.467297 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f058f53-cfb6-4f98-b61b-ce3fbed0cb24 req-80ef9cb5-17f4-4522-b42b-36127707a9f2 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:34.469323 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6f058f53-cfb6-4f98-b61b-ce3fbed0cb24 req-80ef9cb5-17f4-4522-b42b-36127707a9f2 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0806720 May 31 12:00:34.478063 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-5f697ea5-2a49-4285-a414-18626190a8f6 req-77c87c22-ab8d-47ad-b273-6bc3fbd6ba78 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Unset dirty status for tenant:bbcb901221a14602a1f2f8b198287636 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:00:34.479183 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5f697ea5-2a49-4285-a414-18626190a8f6 req-77c87c22-ab8d-47ad-b273-6bc3fbd6ba78 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0974467 May 31 12:00:34.519005 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6f058f53-cfb6-4f98-b61b-ce3fbed0cb24 req-0559bf12-35d5-4ad3-ac1a-39fc39c8ccbe tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0440176 May 31 12:00:34.528530 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5f697ea5-2a49-4285-a414-18626190a8f6 req-54b0df30-d1e0-49bd-a855-78e4042d2b0b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0431669 May 31 12:00:35.718235 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6c3dda16-f9f8-4f0d-82ad-c8aa5231ee4f req-7c984509-2bd5-4ffc-93ac-a50862c6f68f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:35.718852 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ac53bbf-2c47-4834-86ff-647659540f7f req-34678a14-f895-4cfc-bd1a-f1457ae6e654 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:35.719908 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7ac53bbf-2c47-4834-86ff-647659540f7f req-34678a14-f895-4cfc-bd1a-f1457ae6e654 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0812666 May 31 12:00:35.720509 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6c3dda16-f9f8-4f0d-82ad-c8aa5231ee4f req-7c984509-2bd5-4ffc-93ac-a50862c6f68f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0803962 May 31 12:00:35.764669 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7ac53bbf-2c47-4834-86ff-647659540f7f req-67a1c9a7-4d72-4402-800b-12234342ca8b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0390794 May 31 12:00:35.766207 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6c3dda16-f9f8-4f0d-82ad-c8aa5231ee4f req-8d004c91-4dbc-45eb-b7f5-f3285b76716f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0406735 May 31 12:00:36.981871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fcb08872-9338-4c27-93bb-784faaeb419f req-0385298c-e947-43da-87eb-25a79bdf703b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:36.991071 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fcb08872-9338-4c27-93bb-784faaeb419f req-0385298c-e947-43da-87eb-25a79bdf703b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.1140103 May 31 12:00:37.036773 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fcb08872-9338-4c27-93bb-784faaeb419f req-b5deda36-4e98-4a94-bfa7-e3f9af6a16a3 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0448139 May 31 12:00:37.209238 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-577dd4b9-5835-4f42-b52f-f9ce6dfcbf29 req-fb27330b-ebbf-4a4a-9c96-2d3e06319d1a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:37.210690 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-577dd4b9-5835-4f42-b52f-f9ce6dfcbf29 req-fb27330b-ebbf-4a4a-9c96-2d3e06319d1a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0802345 May 31 12:00:37.256267 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-577dd4b9-5835-4f42-b52f-f9ce6dfcbf29 req-8ff4a94b-1ced-431b-9795-da1518ec6463 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0398779 May 31 12:00:38.226517 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c49c00b-34f3-4add-b137-69579b25c2cd req-61d318bf-5f8e-40a3-a56a-fa7cdf34e15f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:38.227920 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3c49c00b-34f3-4add-b137-69579b25c2cd req-61d318bf-5f8e-40a3-a56a-fa7cdf34e15f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0843515 May 31 12:00:38.274396 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3c49c00b-34f3-4add-b137-69579b25c2cd req-eef4904f-645e-4346-914c-55e8295bc952 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0407548 May 31 12:00:38.451793 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72218c11-fdd1-43ec-be47-b9805253a3eb req-2625dab6-cf37-4a78-a8e9-7ea1df35784f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:38.453831 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-72218c11-fdd1-43ec-be47-b9805253a3eb req-2625dab6-cf37-4a78-a8e9-7ea1df35784f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0822759 May 31 12:00:38.498136 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-72218c11-fdd1-43ec-be47-b9805253a3eb req-cac68914-3f67-4af9-a7ed-b7e332621a76 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386386 May 31 12:00:39.037121 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:39.472788 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32f7f876-7c5d-464d-ada0-187379089578 req-0f466ddc-d541-41a2-b6ef-84592739c5aa tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:39.474262 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-32f7f876-7c5d-464d-ada0-187379089578 req-0f466ddc-d541-41a2-b6ef-84592739c5aa tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0931640 May 31 12:00:39.524051 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-32f7f876-7c5d-464d-ada0-187379089578 req-241061c4-1aee-4af8-b6ef-872fcbdd73c6 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0439870 May 31 12:00:39.697792 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ffc4bae-aec7-42b4-84e3-43b87a47d441 req-48918447-bd1d-44b7-a09a-36a3fcc9d61b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:39.699246 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7ffc4bae-aec7-42b4-84e3-43b87a47d441 req-48918447-bd1d-44b7-a09a-36a3fcc9d61b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0857100 May 31 12:00:39.744331 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7ffc4bae-aec7-42b4-84e3-43b87a47d441 req-669fd12b-d073-4bb6-bc49-40adaf1480e4 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389071 May 31 12:00:40.449158 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-43a3f24a-9673-4ed3-97f7-691200161517 req-cc6fc032-b38b-4343-b9b8-d6e6860ee2b0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:40.450635 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-43a3f24a-9673-4ed3-97f7-691200161517 req-cc6fc032-b38b-4343-b9b8-d6e6860ee2b0 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 1098 time: 0.0764713 May 31 12:00:40.601438 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-43a3f24a-9673-4ed3-97f7-691200161517 req-dfca4a26-1d4e-4cf1-a51d-901b993cfc74 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:40.602998 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-43a3f24a-9673-4ed3-97f7-691200161517 req-dfca4a26-1d4e-4cf1-a51d-901b993cfc74 service neutron] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 917 time: 0.1326985 May 31 12:00:40.863051 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-43a3f24a-9673-4ed3-97f7-691200161517 req-164f325b-a51f-4102-ab15-18baadc64389 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=24163e07-db8d-4a12-8f62-285a1abb77fa HTTP/1.1" status: 200 len: 999 time: 0.2533748 May 31 12:00:40.931391 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d28628d9-5af5-4b78-9dae-94f114d73cb1 req-f578b47d-b53d-46be-b353-6ceac1821ef7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:40.932809 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d28628d9-5af5-4b78-9dae-94f114d73cb1 req-f578b47d-b53d-46be-b353-6ceac1821ef7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.2763507 May 31 12:00:40.940268 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9155c9ea-2810-444f-9d51-84e094431545 req-5c6a2a33-7db8-4609-9740-ccab38a90249 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:40.941746 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9155c9ea-2810-444f-9d51-84e094431545 req-5c6a2a33-7db8-4609-9740-ccab38a90249 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0851347 May 31 12:00:40.955059 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-43a3f24a-9673-4ed3-97f7-691200161517 req-640aab43-fc78-4e78-a186-5f8755389f82 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:40.956446 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-43a3f24a-9673-4ed3-97f7-691200161517 req-640aab43-fc78-4e78-a186-5f8755389f82 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0863638 May 31 12:00:40.988197 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9155c9ea-2810-444f-9d51-84e094431545 req-093c95f7-eb33-43f6-9c92-df9eb00348ec tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409071 May 31 12:00:41.023921 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d28628d9-5af5-4b78-9dae-94f114d73cb1 req-f5daed9c-859c-42e3-8dc6-643e432f7dc8 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0846012 May 31 12:00:41.059744 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-43a3f24a-9673-4ed3-97f7-691200161517 req-a1fa00cf-2b86-4fc6-a114-0423de7bbfd9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:41.061190 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-43a3f24a-9673-4ed3-97f7-691200161517 req-a1fa00cf-2b86-4fc6-a114-0423de7bbfd9 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0974896 May 31 12:00:41.192043 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-43a3f24a-9673-4ed3-97f7-691200161517 req-3ed6a9f9-a65c-4015-bfbc-5508bcca8306 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1247053 May 31 12:00:41.322643 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-43a3f24a-9673-4ed3-97f7-691200161517 req-2a61d5e5-08ad-40fc-b89e-c8472954bf16 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1240423 May 31 12:00:42.183375 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8478110f-a25f-471f-b5c3-60cd3757e749 req-cf84a6df-71bd-4794-a75a-195648704b1c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:42.185063 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8478110f-a25f-471f-b5c3-60cd3757e749 req-cf84a6df-71bd-4794-a75a-195648704b1c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0810258 May 31 12:00:42.217708 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-065debe5-2d5e-4127-9367-640c4303b9fe req-643a70e6-d696-47b5-ad4f-151debdb03f4 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:42.219209 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-065debe5-2d5e-4127-9367-640c4303b9fe req-643a70e6-d696-47b5-ad4f-151debdb03f4 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0861406 May 31 12:00:42.235191 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8478110f-a25f-471f-b5c3-60cd3757e749 req-17360ac0-366c-4cd4-a893-895573c7e029 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0430999 May 31 12:00:42.264553 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-065debe5-2d5e-4127-9367-640c4303b9fe req-f5fdf3f8-8d06-4fea-9d8f-a1d18c3cddf1 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0397146 May 31 12:00:42.822308 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5add15c1-9f55-440c-b343-5d2f806a63c2 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:00:42.951412 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:00:42.969510 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'port_id': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'mac_address': 'fa:16:3e:dc:71:13', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 397, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:00:43.030165 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:43.047281 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 5add15c1-9f55-440c-b343-5d2f806a63c2 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:00:43.063454 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:00:43.145384 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:43.147156 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:43.215566 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:43.223555 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.223904 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.224242 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.224576 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.224904 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.225327 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.225553 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.225910 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.226228 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.226549 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.226869 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.227190 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:43.230977 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:43.373361 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['5add15c1-9f55-440c-b343-5d2f806a63c2'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:00:43.430475 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ae3060af-9d16-4a73-84d7-874d05146b6a req-c1fa4834-5446-4de2-bb1b-4852905e8185 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:43.432231 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ae3060af-9d16-4a73-84d7-874d05146b6a req-c1fa4834-5446-4de2-bb1b-4852905e8185 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0851047 May 31 12:00:43.470839 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5016281-8f7b-4dcf-b757-7c07606b8ccd req-5344b052-512a-456a-bf4e-5cf1d7e2c663 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:43.473739 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e5016281-8f7b-4dcf-b757-7c07606b8ccd req-5344b052-512a-456a-bf4e-5cf1d7e2c663 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 944 time: 0.1025348 May 31 12:00:43.484499 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ae3060af-9d16-4a73-84d7-874d05146b6a req-66923c7d-a528-4f97-9081-b771a99fc6da tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0461590 May 31 12:00:43.525469 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e5016281-8f7b-4dcf-b757-7c07606b8ccd req-51397d89-c360-4b21-84d1-89a1054b31a9 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0459254 May 31 12:00:44.688731 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-90009907-0f44-4677-9441-06905c90708b req-9e57de97-de8f-4b1d-8c53-2efed58d2da8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:44.690399 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90009907-0f44-4677-9441-06905c90708b req-9e57de97-de8f-4b1d-8c53-2efed58d2da8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0872414 May 31 12:00:44.735725 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90009907-0f44-4677-9441-06905c90708b req-c2ffe322-fb66-44c3-abb9-bb0eac7aa376 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0392289 May 31 12:00:44.739729 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-713a2800-63a2-45fc-9f54-80d5100b8417 req-ea1ee66a-6b17-4f2a-9571-a271d8c07de2 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:44.745885 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-713a2800-63a2-45fc-9f54-80d5100b8417 req-ea1ee66a-6b17-4f2a-9571-a271d8c07de2 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 944 time: 0.1023376 May 31 12:00:44.787652 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-713a2800-63a2-45fc-9f54-80d5100b8417 req-53c2f14c-e50e-4078-b8de-251751841cf0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0406663 May 31 12:00:44.825525 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5add15c1-9f55-440c-b343-5d2f806a63c2 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:00:44.991137 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 5add15c1-9f55-440c-b343-5d2f806a63c2 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:00:44.996212 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 5add15c1-9f55-440c-b343-5d2f806a63c2 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:00:44.996591 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:45.108977 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:45.137661 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:00:45.140644 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-25e5d290-4c0e-4739-bf7d-b79e7f5d93ed" -d '{"events": [{"server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "name": "network-vif-plugged", "status": "completed", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:00:45.148321 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:00:45.248214 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:45.251085 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:45.316789 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:45.319838 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:45.371823 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:00:45 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-2afc03ba-2865-4bef-92f1-b0553d0c567e x-openstack-request-id: req-2afc03ba-2865-4bef-92f1-b0553d0c567e {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:00:45.372127 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "name": "network-vif-plugged", "status": "completed", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:00:45.372390 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-2afc03ba-2865-4bef-92f1-b0553d0c567e {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:00:45.372841 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'code': 200} May 31 12:00:45.837603 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69a02035-0604-4a44-abc9-d32066afd817 req-9dbeb472-6c50-45bd-a958-4c59370ebeba service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:45.839107 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-69a02035-0604-4a44-abc9-d32066afd817 req-9dbeb472-6c50-45bd-a958-4c59370ebeba service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1098 time: 0.0723548 May 31 12:00:45.944494 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-69a02035-0604-4a44-abc9-d32066afd817 req-336cbf3b-1ce3-47d6-8bf9-de21d66846d2 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0983398 May 31 12:00:45.967705 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce8e5c14-e8bb-4576-8265-77c8c7fe635a req-f5044d4a-f292-4f31-9e65-d2a1cba74008 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:45.969220 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce8e5c14-e8bb-4576-8265-77c8c7fe635a req-f5044d4a-f292-4f31-9e65-d2a1cba74008 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.1060362 May 31 12:00:45.993387 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-720500eb-5590-4136-ab55-abd1f0cecc30 req-6670d84d-f991-4021-9cc4-c8a09ecf2691 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:45.995515 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-720500eb-5590-4136-ab55-abd1f0cecc30 req-6670d84d-f991-4021-9cc4-c8a09ecf2691 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0856416 May 31 12:00:46.018556 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce8e5c14-e8bb-4576-8265-77c8c7fe635a req-01553024-dfe4-43e1-a732-89a306eb89a1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0420685 May 31 12:00:46.041222 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-720500eb-5590-4136-ab55-abd1f0cecc30 req-2701dfda-5185-415a-869a-b952b3349b29 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0394077 May 31 12:00:47.212637 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-501786c5-e376-4e88-b177-6788bcca4b90 req-0e30cebf-bd15-4380-9afe-a4d2bdde0888 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:47.214081 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-501786c5-e376-4e88-b177-6788bcca4b90 req-0e30cebf-bd15-4380-9afe-a4d2bdde0888 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0864096 May 31 12:00:47.236066 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4db0d2b2-e714-44fa-9c5c-3a6a66428a03 req-b5738184-117b-4877-8827-add527a0b4e2 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:47.237532 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4db0d2b2-e714-44fa-9c5c-3a6a66428a03 req-b5738184-117b-4877-8827-add527a0b4e2 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0870938 May 31 12:00:47.258610 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-501786c5-e376-4e88-b177-6788bcca4b90 req-8dc3c778-7478-481a-a978-4f580c301ca0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0385289 May 31 12:00:47.283625 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4db0d2b2-e714-44fa-9c5c-3a6a66428a03 req-15d96a2a-44ae-4a48-a462-eefc2bff6379 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0401740 May 31 12:00:47.469208 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6feb55cb-ce33-4e78-ba2e-ac75fb22b077 req-26f8a795-c000-4896-90f9-436237142683 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:47.470655 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6feb55cb-ce33-4e78-ba2e-ac75fb22b077 req-26f8a795-c000-4896-90f9-436237142683 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0779495 May 31 12:00:47.525524 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6feb55cb-ce33-4e78-ba2e-ac75fb22b077 req-21109870-4787-4441-90e5-da392aeb6344 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0490496 May 31 12:00:47.549029 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37712) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:47.785141 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9bce34e2-d0c7-4b43-a319-cf6033b45351 tempest-TestShelveInstance-1420395607 tempest-TestShelveInstance-1420395607-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:47.785867 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-9bce34e2-d0c7-4b43-a319-cf6033b45351 tempest-TestShelveInstance-1420395607 tempest-TestShelveInstance-1420395607-project-admin] Synchronizing usage tracker for tenant:00afe2f4c68c431385f2a4b6061b0f72 on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:00:47.865429 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-9bce34e2-d0c7-4b43-a319-cf6033b45351 tempest-TestShelveInstance-1420395607 tempest-TestShelveInstance-1420395607-project-admin] Unset dirty status for tenant:00afe2f4c68c431385f2a4b6061b0f72 on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:00:47.866541 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-9bce34e2-d0c7-4b43-a319-cf6033b45351 tempest-TestShelveInstance-1420395607 tempest-TestShelveInstance-1420395607-project-admin] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1093 time: 0.3164372 May 31 12:00:47.869429 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37722) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:00:47.874361 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'port_id': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'fixed_ip_address': '172.20.0.6'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:00:47.882045 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:00:47.884114 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Usage tracker for resource:floatingip and tenant:bbcb901221a14602a1f2f8b198287636 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:00:47.886990 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:00:47.893262 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:00:48.025393 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for floatingip, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:48.238519 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:48.381549 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Allocated IP 172.27.1.240 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/babc9027-ff78-4d33-b481-f3e9ce412dfc) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:00:48.455945 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca0b6ec7-a34d-46d2-8005-e66381dd36f1 req-89ae0400-4cec-4bb0-8719-237acd3c319b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:48.457390 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca0b6ec7-a34d-46d2-8005-e66381dd36f1 req-89ae0400-4cec-4bb0-8719-237acd3c319b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0861537 May 31 12:00:48.507545 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ca0b6ec7-a34d-46d2-8005-e66381dd36f1 req-c756d89f-d866-4feb-91d1-a07b140a8963 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0441687 May 31 12:00:48.535494 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:48.619675 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:48.625014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fe3c08fe-2cb8-4836-85ac-3950fcb23594 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:48.797107 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:00:48.800404 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:00:48.803811 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:00:48.833998 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'Port': ['ID=babc9027-ff78-4d33-b481-f3e9ce412dfc,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:48.848057 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.848515 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.848937 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.849358 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.849800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.850247 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.850726 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.851262 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.851802 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.852183 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.852575 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.852976 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:00:48.864106 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.240s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:49.253818 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:49.440574 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:49.467624 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:00:49.657837 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:49.696800 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6efe9e5e-b919-45da-a44a-2404196ec416 req-89cd7fb8-c44f-4c3f-a845-8a9b9c80a305 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:49.698270 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6efe9e5e-b919-45da-a44a-2404196ec416 req-89cd7fb8-c44f-4c3f-a845-8a9b9c80a305 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0794396 May 31 12:00:49.722477 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:49.727250 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:49.751259 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6efe9e5e-b919-45da-a44a-2404196ec416 req-6e7f0f28-1c34-454e-987c-01a96bfebbb0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0467978 May 31 12:00:49.834726 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event updated for resources: {'Port': ['ID=babc9027-ff78-4d33-b481-f3e9ce412dfc,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:49.838797 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:49.843259 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:50.008281 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.l3_db [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Floating IP d49963a0-0633-44de-951e-20f07f82e459 associated. External IP: 172.27.1.240, port: 5add15c1-9f55-440c-b343-5d2f806a63c2. May 31 12:00:50.346781 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Router 0da52ba7-9067-4610-8053-ced8a908f89b has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:00:50.623267 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:00:50.632596 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] Sync routers for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:00:50.642056 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:00:50.642682 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:00:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:50.768030 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:50.832411 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-83a7e67e-906d-46f8-aabd-b7cc1cb45dbb tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 2.9619222 May 31 12:00:50.834429 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:00:50.835482 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-92316553-6b69-4f37-90c7-69c9f032e6bd" -d '{"events": [{"name": "network-changed", "server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:00:50.875459 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:00:50.952566 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:00:50 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-4a6ef560-4c30-4805-add2-cddf42910f8e x-openstack-request-id: req-4a6ef560-4c30-4805-add2-cddf42910f8e {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:00:50.952967 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:00:50.953312 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-4a6ef560-4c30-4805-add2-cddf42910f8e {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:00:50.953801 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'status': 'completed', 'code': 200} May 31 12:00:50.975743 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-efde8660-ff35-480f-8614-4a94d01934be req-5470a7ab-e1d5-4c33-bde1-d99dcc8d75cc tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:50.982040 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-efde8660-ff35-480f-8614-4a94d01934be req-5470a7ab-e1d5-4c33-bde1-d99dcc8d75cc tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.1052387 May 31 12:00:51.030778 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-efde8660-ff35-480f-8614-4a94d01934be req-75f0f41d-3248-472c-856f-14cbf100c4e8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475450 May 31 12:00:51.043974 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3ccb2cba-949b-4b7e-a771-79bcd1edf714 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:51.058260 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-1b4920ac-6c31-411b-9095-164a46dbd99d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:51.059786 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-1b4920ac-6c31-411b-9095-164a46dbd99d service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1098 time: 0.0859432 May 31 12:00:51.207948 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-a8d631df-23c2-4cff-abfc-8fe73bf79b42 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:51.209401 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-a8d631df-23c2-4cff-abfc-8fe73bf79b42 service neutron] 10.222.0.47 "GET /v2.0/networks?id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 200 len: 910 time: 0.1426108 May 31 12:00:51.282204 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'd49963a0-0633-44de-951e-20f07f82e459', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'floating_ip_address': '172.27.1.240', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'port_id': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 549, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'mac_address': 'fa:16:3e:dc:71:13', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:00:49Z', 'updated_at': '2021-05-31T12:00:49Z', 'revision_number': 0, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:57Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:51.286391 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] Checking router: 0da52ba7-9067-4610-8053-ced8a908f89b for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:00:51.295251 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:00:51.296453 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:00:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:51.399240 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-4e4c7f20-64ca-4e20-b013-e8f1a0160f1a service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 200 len: 999 time: 0.1832590 May 31 12:00:51.461397 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-afe53594-e0d2-4651-9489-8ad2ceddf5fb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:51.462777 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-afe53594-e0d2-4651-9489-8ad2ceddf5fb service neutron] 10.222.0.47 "GET /v2.0/subnets?id=3a19e015-3084-4732-bb65-ea124df3118f HTTP/1.1" status: 200 len: 848 time: 0.0566332 May 31 12:00:51.475643 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:00:51.548576 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-a38d7443-a193-42b9-9b52-6db2f951f48f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:51.550053 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-a38d7443-a193-42b9-9b52-6db2f951f48f service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0809181 May 31 12:00:51.688515 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-0481790e-aac7-46fc-b241-d2f697ca25bb service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1317656 May 31 12:00:51.820300 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-92316553-6b69-4f37-90c7-69c9f032e6bd req-76af376d-0c2b-4138-8daf-ae7c90706799 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1254094 May 31 12:00:51.833098 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'd49963a0-0633-44de-951e-20f07f82e459', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'floating_ip_address': '172.27.1.240', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'port_id': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 549, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'mac_address': 'fa:16:3e:dc:71:13', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:00:49Z', 'updated_at': '2021-05-31T12:00:49Z', 'revision_number': 0, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:57Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:00:51.837179 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-31090914-eca2-4c0d-b163-9da6c810faed None None] The sync data for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi is: [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'd49963a0-0633-44de-951e-20f07f82e459', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'floating_ip_address': '172.27.1.240', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'port_id': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 549, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'mac_address': 'fa:16:3e:dc:71:13', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:00:49Z', 'updated_at': '2021-05-31T12:00:49Z', 'revision_number': 0, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:57Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:00:52.034432 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f15e8b8c-c5c9-4149-a07c-5b4948c5da1a None None] New status for floating IP d49963a0-0633-44de-951e-20f07f82e459: ACTIVE {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:00:52.236910 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-26f5958b-da3c-4210-ae15-f7df38664e97 req-9de79937-952b-492b-a683-dde3ea09e9d3 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:52.238396 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-26f5958b-da3c-4210-ae15-f7df38664e97 req-9de79937-952b-492b-a683-dde3ea09e9d3 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0846162 May 31 12:00:52.283475 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-26f5958b-da3c-4210-ae15-f7df38664e97 req-1f05ea2a-2509-46bf-9252-3f2c88b6c731 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0391910 May 31 12:00:53.368369 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:53.487959 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-7c911f42-6652-4fc0-9e8a-f08a04bd4324 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:53.492177 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-be366ee4-f89d-4995-a12d-c3f2cc8efb39 req-531d33ae-0f7b-4d83-9ece-3616f65a709f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:53.493691 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-be366ee4-f89d-4995-a12d-c3f2cc8efb39 req-531d33ae-0f7b-4d83-9ece-3616f65a709f tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0832117 May 31 12:00:53.544545 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-be366ee4-f89d-4995-a12d-c3f2cc8efb39 req-aea909a7-d33e-429c-97c7-9fcb58ed17f5 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0442338 May 31 12:00:54.596366 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:54.734463 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-607dd1a5-e928-41ac-9dcd-a7114316b965 req-76871128-3e05-4af3-a6d2-55979abcd18e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:54.735890 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-607dd1a5-e928-41ac-9dcd-a7114316b965 req-76871128-3e05-4af3-a6d2-55979abcd18e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0828841 May 31 12:00:54.782988 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-607dd1a5-e928-41ac-9dcd-a7114316b965 req-7d70b819-335b-42c6-ab9c-dbff5891f42e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414026 May 31 12:00:55.978924 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f44884b-a0f5-4bda-93e1-48cd6fd66aaa req-3d1952f9-9999-4757-9b20-c1ef2632c093 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:55.980538 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6f44884b-a0f5-4bda-93e1-48cd6fd66aaa req-3d1952f9-9999-4757-9b20-c1ef2632c093 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0840704 May 31 12:00:56.026293 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6f44884b-a0f5-4bda-93e1-48cd6fd66aaa req-9d51f7d4-af26-40fb-99e4-941f21515066 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0394702 May 31 12:00:56.122484 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:57.227544 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1c3ffe0-5d81-46e1-8650-e3b7f547b957 req-b2dc75b9-e0ff-4d8b-9696-db0f62ec5806 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:57.228990 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d1c3ffe0-5d81-46e1-8650-e3b7f547b957 req-b2dc75b9-e0ff-4d8b-9696-db0f62ec5806 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0869410 May 31 12:00:57.281739 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d1c3ffe0-5d81-46e1-8650-e3b7f547b957 req-02790f79-dbb8-4527-9b16-6447037f5fbb tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0464046 May 31 12:00:58.480695 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6def953-90b6-4ecf-9367-fe4400fb6456 req-8ffe68ba-adfb-4052-8b53-7c6e8dc9d5bb tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:58.482158 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f6def953-90b6-4ecf-9367-fe4400fb6456 req-8ffe68ba-adfb-4052-8b53-7c6e8dc9d5bb tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0813611 May 31 12:00:58.528425 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f6def953-90b6-4ecf-9367-fe4400fb6456 req-59584566-afa1-4b6a-ae8a-13f224dc3599 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0397601 May 31 12:00:58.925706 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:00:59.056699 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:00:59.069790 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'port_id': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'mac_address': 'fa:16:3e:6f:6d:78', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 371, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:00:59.126154 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:59.142908 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id 24163e07-db8d-4a12-8f62-285a1abb77fa {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:00:59.157912 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:00:59.242002 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:00:59.244504 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:00:59.313403 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=6']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:00:59.316233 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:00:59.353945 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['24163e07-db8d-4a12-8f62-285a1abb77fa'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:00:59.723517 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-02d01687-f764-4b91-82c2-2784a6af7244 req-fffaefb0-a58f-48a0-b102-96771f89bc69 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:00:59.724976 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-02d01687-f764-4b91-82c2-2784a6af7244 req-fffaefb0-a58f-48a0-b102-96771f89bc69 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 944 time: 0.0839050 May 31 12:00:59.769515 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-02d01687-f764-4b91-82c2-2784a6af7244 req-5e452126-26a5-46d1-a993-306d188109cd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389106 May 31 12:01:00.950163 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:01:01.032218 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 24163e07-db8d-4a12-8f62-285a1abb77fa triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:01:01.032690 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:01.153597 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b1d34c46-4082-48d2-bd2e-5060ec37fb29 req-642028f0-7c42-4a4f-8594-9d2df7073f28 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:01.162554 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b1d34c46-4082-48d2-bd2e-5060ec37fb29 req-642028f0-7c42-4a4f-8594-9d2df7073f28 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 944 time: 0.1360824 May 31 12:01:01.215644 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b1d34c46-4082-48d2-bd2e-5060ec37fb29 req-a68f92fe-b38e-493b-a23a-f295842b0d80 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0520573 May 31 12:01:01.230062 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:01.275559 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:01:01.285413 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-3b3b6fcc-afc3-4243-b2cb-4c96b93c4573" -d '{"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-plugged", "status": "completed", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:01:01.294223 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:01.384245 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:01:01 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-d94f7eb0-2fdc-4693-b8c3-f0bb2803d406 x-openstack-request-id: req-d94f7eb0-2fdc-4693-b8c3-f0bb2803d406 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:01:01.384245 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-plugged", "status": "completed", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:01:01.384245 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-d94f7eb0-2fdc-4693-b8c3-f0bb2803d406 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:01:01.384245 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'code': 200} May 31 12:01:01.408032 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:01.410261 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:01.487052 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=7']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:01.501680 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:02.433364 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ffce3d5-33dc-4b1e-9a8b-100406f21eb1 req-3657e603-aa7d-4c13-8f60-15f2f4454e12 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:02.435617 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ffce3d5-33dc-4b1e-9a8b-100406f21eb1 req-3657e603-aa7d-4c13-8f60-15f2f4454e12 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0862565 May 31 12:01:02.452892 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-02560efb-b4e1-49d6-8278-14ac3eb8d556 req-1f08a69f-9459-4288-b6ec-f9bd170d7dc6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:02.454396 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-02560efb-b4e1-49d6-8278-14ac3eb8d556 req-1f08a69f-9459-4288-b6ec-f9bd170d7dc6 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 946 time: 0.0967062 May 31 12:01:02.508946 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7ffce3d5-33dc-4b1e-9a8b-100406f21eb1 req-2133f7b2-0c41-4232-b3bf-8b1dfd1b61b7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0612192 May 31 12:01:02.528902 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-02560efb-b4e1-49d6-8278-14ac3eb8d556 req-de7af422-7134-44de-9f4c-2a0d98d11f19 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0685530 May 31 12:01:03.720738 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db4d504e-1584-4c4c-a654-57c24285c3f9 req-39c8effa-2d05-49fc-bc49-1cd01fac9e86 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:03.722526 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db4d504e-1584-4c4c-a654-57c24285c3f9 req-39c8effa-2d05-49fc-bc49-1cd01fac9e86 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0929613 May 31 12:01:03.726388 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78ba4ece-2cf5-49c2-a704-2554f5642dac req-547b383f-f98d-41a2-b0ad-795cbcf40849 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:03.728637 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-78ba4ece-2cf5-49c2-a704-2554f5642dac req-547b383f-f98d-41a2-b0ad-795cbcf40849 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 946 time: 0.0889192 May 31 12:01:03.771870 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db4d504e-1584-4c4c-a654-57c24285c3f9 req-bedef260-1aaf-4dd7-8a5e-ed2d928f3343 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0427935 May 31 12:01:03.790244 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-78ba4ece-2cf5-49c2-a704-2554f5642dac req-5f12b8ad-ad30-4018-8bd4-7aa3cf4181dd tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0460680 May 31 12:01:04.152516 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-049c46fc-f966-4eda-a6b4-10bdd22b9fef tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:04.154030 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-049c46fc-f966-4eda-a6b4-10bdd22b9fef tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 946 time: 0.0817409 May 31 12:01:04.295490 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Deleting port 6bd1051c-af78-4616-88bd-002db4d1b521 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:01:04.296048 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:04.452129 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] For port 6bd1051c-af78-4616-88bd-002db4d1b521, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6bd1051c-af78-4616-88bd-002db4d1b521,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:04.454057 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:04.931378 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:04.932103 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:04.932103 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Floating IP 35e7f9d3-2578-4db6-a21c-2bc73f629dc0 disassociated. External IP: 172.27.1.248, port: 6bd1051c-af78-4616-88bd-002db4d1b521. May 31 12:01:04.932471 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Calling delete_port for 6bd1051c-af78-4616-88bd-002db4d1b521 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:01:05.003016 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1344ad2-d06e-4273-8a55-685938e4c4d2 req-e6719097-b712-4a46-90ad-3efe2740d940 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:05.011026 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e1344ad2-d06e-4273-8a55-685938e4c4d2 req-e6719097-b712-4a46-90ad-3efe2740d940 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 946 time: 0.1008835 May 31 12:01:05.066809 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e1344ad2-d06e-4273-8a55-685938e4c4d2 req-8425bd59-bc9a-4c84-9334-aa2de4baee40 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups?id=237a0cf4-5be9-4442-a0d4-170bc2d699a7&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0542731 May 31 12:01:05.146676 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f5ec4b15-b0e9-46e0-8393-dc4159a7156a req-8058715d-dfc5-47ef-beed-0473624844ee tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:05.148298 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f5ec4b15-b0e9-46e0-8393-dc4159a7156a req-8058715d-dfc5-47ef-beed-0473624844ee tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0966206 May 31 12:01:05.175514 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:05.189185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1404accd-2664-409d-a863-da64b734d81a tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:05.280023 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event deleted for resources: {'Port': ['ID=6bd1051c-af78-4616-88bd-002db4d1b521,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:05.296764 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f5ec4b15-b0e9-46e0-8393-dc4159a7156a req-ce897436-2bb5-46f5-9cb8-24268ff46e2d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1425354 May 31 12:01:05.299849 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:05.521559 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Router 6c5aae18-4faa-4947-af35-4078d428c2b9 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:01:05.735846 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:01:05.748486 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] Sync routers for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:01:05.765819 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:01:05.766694 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:00:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:05.844073 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] No DVR routers for this DVR port 6bd1051c-af78-4616-88bd-002db4d1b521 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:01:05.848134 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:05.851520 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'name': 'network-vif-deleted', 'tag': '6bd1051c-af78-4616-88bd-002db4d1b521'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:01:05.854846 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-a4efd9d5-0932-4a97-b422-27dfe990b34c" -d '{"events": [{"server_uuid": "dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca", "name": "network-vif-deleted", "tag": "6bd1051c-af78-4616-88bd-002db4d1b521"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:01:05.914044 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:01:05 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-2790b3ed-fb3a-4438-9aeb-9371fc6b2160 x-openstack-request-id: req-2790b3ed-fb3a-4438-9aeb-9371fc6b2160 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:01:05.914044 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca", "name": "network-vif-deleted", "tag": "6bd1051c-af78-4616-88bd-002db4d1b521", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:01:05.914044 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-2790b3ed-fb3a-4438-9aeb-9371fc6b2160 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:01:05.914044 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca', 'name': 'network-vif-deleted', 'tag': '6bd1051c-af78-4616-88bd-002db4d1b521', 'status': 'completed', 'code': 200} May 31 12:01:05.980129 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b9ee5075-c07c-43fa-86b6-21cd0a434ba9 req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.47 "DELETE /v2.0/ports/6bd1051c-af78-4616-88bd-002db4d1b521 HTTP/1.1" status: 204 len: 173 time: 1.8203700 May 31 12:01:06.001664 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:01:06.246612 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-af2ecfc0-2b5e-45ba-b690-1f62a99a109e req-4307732c-681e-4828-a02c-b5192850160e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Synchronizing usage tracker for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:01:06.363987 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-af2ecfc0-2b5e-45ba-b690-1f62a99a109e req-4307732c-681e-4828-a02c-b5192850160e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Unset dirty status for tenant:b866e6eab1b04406a7280f91698983ce on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:01:06.365356 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-af2ecfc0-2b5e-45ba-b690-1f62a99a109e req-4307732c-681e-4828-a02c-b5192850160e tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/ports?device_id=dcb1b25a-55e1-4468-ae0a-2a48dbb7a8ca HTTP/1.1" status: 200 len: 210 time: 0.1697733 May 31 12:01:06.462544 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:06.466343 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] Checking router: 6c5aae18-4faa-4947-af35-4078d428c2b9 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:01:06.475250 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:01:06.475855 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:00:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:06.558695 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c7503dc-8930-4b01-9cb0-69410011cddf req-c581297d-b109-4513-8277-708fd411aa4a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:06.568239 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2c7503dc-8930-4b01-9cb0-69410011cddf req-c581297d-b109-4513-8277-708fd411aa4a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.1202490 May 31 12:01:06.640829 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2c7503dc-8930-4b01-9cb0-69410011cddf req-6b6126ef-5ef1-4413-b15d-271cdf1357c1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0685794 May 31 12:01:06.683042 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:01:07.055110 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:07.060792 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9d1ae861-8b73-4b04-b046-3e349b12e344 None None] The sync data for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi is: [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:27Z', 'revision_number': 4, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': 'b5af8b49-c5f1-494f-aa5a-3d080497d163', 'name': '', 'network_id': '36cce22a-502c-464d-a9ac-50f2bc247e07', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'mac_address': 'fa:16:3e:33:19:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_interface', 'standard_attr_id': 99, 'fixed_ips': [{'subnet_id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:24Z', 'updated_at': '2021-05-31T11:54:46Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'subnets': [{'id': '1b74adba-cf2f-4db5-ad92-27ea39726767', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:01:07.402739 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a8c2263-8200-4078-b24f-3e9b19d0a252 None None] New status for floating IP 35e7f9d3-2578-4db6-a21c-2bc73f629dc0: DOWN {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:01:07.586277 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37876) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:07.660418 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bf085303-a12d-4706-99e3-cbe265f01140 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:07.702179 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bf085303-a12d-4706-99e3-cbe265f01140 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:07.853556 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bf085303-a12d-4706-99e3-cbe265f01140 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:07.859503 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:07.884428 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf085303-a12d-4706-99e3-cbe265f01140 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=237a0cf4-5be9-4442-a0d4-170bc2d699a7,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:07.891310 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bf085303-a12d-4706-99e3-cbe265f01140 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:07.917382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-405f417d-7aa1-4db6-86d4-cde229da3c91 req-53688e5c-2f23-4bf0-8503-b43eb1496465 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:07.924712 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-405f417d-7aa1-4db6-86d4-cde229da3c91 req-53688e5c-2f23-4bf0-8503-b43eb1496465 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.1468577 May 31 12:01:07.978560 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-405f417d-7aa1-4db6-86d4-cde229da3c91 req-9cb1c4aa-1865-4d2e-9b0a-441f0e8053c0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0529809 May 31 12:01:07.981972 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bf085303-a12d-4706-99e3-cbe265f01140 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:07.987072 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-bf085303-a12d-4706-99e3-cbe265f01140 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "DELETE /v2.0/security-groups/237a0cf4-5be9-4442-a0d4-170bc2d699a7 HTTP/1.1" status: 204 len: 168 time: 0.3958793 May 31 12:01:07.987600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37886) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:08.013456 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.api.v2.resource [None req-609d2d6f-17c0-4972-afef-6bcc2dfb2da4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] show failed (client error): The resource could not be found. May 31 12:01:08.014570 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-609d2d6f-17c0-4972-afef-6bcc2dfb2da4 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "GET /v2.0/security-groups/237a0cf4-5be9-4442-a0d4-170bc2d699a7 HTTP/1.1" status: 404 len: 347 time: 0.0252612 May 31 12:01:08.017671 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 37890) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:08.288354 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Deleting port 199cf35d-e696-43da-9032-c8ab40b0707d {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:01:08.288784 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:08.416422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:08.416730 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:08.646508 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Calling delete_port for 199cf35d-e696-43da-9032-c8ab40b0707d owned by network:floatingip {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:01:08.830675 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:08.836292 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a8ac4b7a-bc79-4c2c-bc3c-954b50b08c00 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:08.884167 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Pushing event deleted for resources: {'Port': ['ID=199cf35d-e696-43da-9032-c8ab40b0707d,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:08.888446 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:08.987610 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks [] for floatingip, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:09.003252 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 6bd1051c-af78-4616-88bd-002db4d1b521 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:01:09.056122 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 6bd1051c-af78-4616-88bd-002db4d1b521 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:01:09.056596 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 6bd1051c-af78-4616-88bd-002db4d1b521 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:01:09.119300 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:09.220718 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:09.221085 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6bcc56a7-b934-4ed2-9e6b-b676f57e0665 req-ad779324-e604-4caa-a409-b0a47786e809 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:09.222480 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6bcc56a7-b934-4ed2-9e6b-b676f57e0665 req-ad779324-e604-4caa-a409-b0a47786e809 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.1147578 May 31 12:01:09.226951 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] 10.222.0.48 "DELETE /v2.0/floatingips/35e7f9d3-2578-4db6-a21c-2bc73f629dc0 HTTP/1.1" status: 204 len: 168 time: 1.2072144 May 31 12:01:09.275615 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6bcc56a7-b934-4ed2-9e6b-b676f57e0665 req-bd2beefb-4e43-45f2-8fd4-98d7bdbb55c6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0463977 May 31 12:01:10.477748 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a38620e-c663-4f03-9be8-b0e163972762 req-beda1b16-0395-44fe-8eed-430208cba66c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:10.479047 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4a38620e-c663-4f03-9be8-b0e163972762 req-beda1b16-0395-44fe-8eed-430208cba66c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0826385 May 31 12:01:10.533795 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4a38620e-c663-4f03-9be8-b0e163972762 req-bd95eb04-8af9-4313-b678-6b2ece671927 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0418816 May 31 12:01:11.729051 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55036333-517c-4685-b5ae-c1d857684b08 req-cf81fd69-6edb-4c03-ae99-f3ecf6f0803b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:11.730493 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-55036333-517c-4685-b5ae-c1d857684b08 req-cf81fd69-6edb-4c03-ae99-f3ecf6f0803b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0868304 May 31 12:01:11.775745 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-55036333-517c-4685-b5ae-c1d857684b08 req-0afcfe7e-1c94-413a-ba17-f1f1bdf8c72d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0385978 May 31 12:01:12.573107 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-e091e19e-fab3-40ae-ba79-321c10039095 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:12.574690 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-e091e19e-fab3-40ae-ba79-321c10039095 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 1098 time: 0.0742061 May 31 12:01:12.709202 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-e3defb0b-8e54-412a-bd32-f543ec77116a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:12.710704 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-e3defb0b-8e54-412a-bd32-f543ec77116a service neutron] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 917 time: 0.1293085 May 31 12:01:12.814290 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-d54a4e10-dab9-4ee8-aa8c-47a720838f55 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=24163e07-db8d-4a12-8f62-285a1abb77fa HTTP/1.1" status: 200 len: 999 time: 0.0971024 May 31 12:01:12.880358 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-af8913d2-26c1-4e11-9d35-3f6e8aa16292 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:12.881732 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-af8913d2-26c1-4e11-9d35-3f6e8aa16292 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0609391 May 31 12:01:13.016313 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-11861a8d-4ad5-4d33-973d-1f2ee4087628 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:13.017780 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-11861a8d-4ad5-4d33-973d-1f2ee4087628 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1287305 May 31 12:01:13.025714 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de39a01a-9cd0-4af7-b61a-06d76eb68fbb req-b7a5596c-d4bd-499b-86e3-834eedaa56a6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:13.027030 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-de39a01a-9cd0-4af7-b61a-06d76eb68fbb req-b7a5596c-d4bd-499b-86e3-834eedaa56a6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.1396468 May 31 12:01:13.148756 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-de39a01a-9cd0-4af7-b61a-06d76eb68fbb req-f0171eda-6ca4-45a8-b45b-e83924447cc6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1040635 May 31 12:01:13.184411 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-2b24d211-8e76-4f18-93cf-31c857586b5a service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1564574 May 31 12:01:13.315255 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0ea7d837-df4b-4582-874a-d5433ddb4bed req-74e59207-8fb9-4e73-9435-ff1dda5ba34c service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1228559 May 31 12:01:13.858514 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37944) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:14.539335 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-929a0ea3-546d-4034-819b-ae603ab7ec12 req-83876d26-5607-4e57-9812-87c972b471c3 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:14.540748 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-929a0ea3-546d-4034-819b-ae603ab7ec12 req-83876d26-5607-4e57-9812-87c972b471c3 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.2751250 May 31 12:01:14.602556 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-929a0ea3-546d-4034-819b-ae603ab7ec12 req-7880b99a-0787-46d1-b142-80ba67a200a8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0554137 May 31 12:01:14.685836 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:15.084930 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Deleting port b5af8b49-c5f1-494f-aa5a-3d080497d163 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:01:15.085627 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:15.204554 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] For port b5af8b49-c5f1-494f-aa5a-3d080497d163, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b5af8b49-c5f1-494f-aa5a-3d080497d163,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:15.204928 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:15.331203 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Calling delete_port for b5af8b49-c5f1-494f-aa5a-3d080497d163 owned by network:router_interface {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:01:15.682249 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:15.687698 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:15.731738 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Pushing event deleted for resources: {'Port': ['ID=b5af8b49-c5f1-494f-aa5a-3d080497d163,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:15.764779 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:15.814734 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6c2c625-8f13-4ce0-b987-cf94ec9bded7 req-0efd61f8-86b5-4230-a5c7-548c244b385b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:15.816295 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a6c2c625-8f13-4ce0-b987-cf94ec9bded7 req-0efd61f8-86b5-4230-a5c7-548c244b385b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 945 time: 0.0993066 May 31 12:01:15.864798 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a6c2c625-8f13-4ce0-b987-cf94ec9bded7 req-f7507297-3c75-46f1-813a-e063f2af8d78 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0419335 May 31 12:01:16.122800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:16.484367 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Router 6c5aae18-4faa-4947-af35-4078d428c2b9 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:01:16.663882 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:01:16.673501 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] Sync routers for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:01:16.678211 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] 10.222.0.48 "PUT /v2.0/routers/6c5aae18-4faa-4947-af35-4078d428c2b9/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.8185208 May 31 12:01:16.680647 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37962) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:16.690959 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:01:16.693667 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:00:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:16.863665 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks [] for router, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:16.913352 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:01:17.036601 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:17.093772 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:01:17.220840 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Deleting port 16b2c21e-cb05-472a-b273-fe85f07e77cc {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:01:17.221327 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:17.290621 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:17.332675 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:01:17.334909 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-0ef704ca-aa46-4963-86b0-36b46e406349" -d '{"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-unplugged", "status": "completed", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:01:17.347313 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] For port 16b2c21e-cb05-472a-b273-fe85f07e77cc, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=16b2c21e-cb05-472a-b273-fe85f07e77cc,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:17.348014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:17.349140 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122'}], 'enable_snat': True}, 'gw_port_id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T12:01:15Z', 'revision_number': 5, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port': {'id': '16b2c21e-cb05-472a-b273-fe85f07e77cc', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:c7:1e:fd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 86, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.122', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:17.352024 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:17.355326 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] Checking router: 6c5aae18-4faa-4947-af35-4078d428c2b9 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:01:17.365662 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] Routers {'6c5aae18-4faa-4947-af35-4078d428c2b9'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:01:17.366435 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:00:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['6c5aae18-4faa-4947-af35-4078d428c2b9'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:17.425193 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [207] Connection: close Content-Length: 182 Content-Type: application/json Date: Mon, 31 May 2021 12:01:17 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-14cfbb83-ee40-460a-bb22-0c02291c432a x-openstack-request-id: req-14cfbb83-ee40-460a-bb22-0c02291c432a {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:01:17.426610 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-unplugged", "status": "failed", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa", "code": 422}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:01:17.426610 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-14cfbb83-ee40-460a-bb22-0c02291c432a {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:01:17.426610 dv-cinder-791239-4-iscsi neutron-server[99041]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'code': 422} returned with failed status May 31 12:01:17.486814 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:17.495846 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:17.589885 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] Set router 6c5aae18-4faa-4947-af35-4078d428c2b9 gateway port host: None {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:01:17.590368 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Calling delete_port for 16b2c21e-cb05-472a-b273-fe85f07e77cc owned by network:router_gateway {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:01:17.707017 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=8']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:17.710304 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T12:01:17Z', 'revision_number': 6, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port_host': None}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:17.712589 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.217s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:17.715346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fea8306b-8e74-4cd6-abb2-01ad7d063416 None None] The sync data for ids ['6c5aae18-4faa-4947-af35-4078d428c2b9'] in dv-cinder-791239-4-iscsi is: [{'id': '6c5aae18-4faa-4947-af35-4078d428c2b9', 'name': 'tempest-AttachVolumeTestJSON-1240663657-router', 'tenant_id': 'b866e6eab1b04406a7280f91698983ce', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T12:01:17Z', 'revision_number': 6, 'project_id': 'b866e6eab1b04406a7280f91698983ce', 'gw_port_host': None}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:01:17.807961 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:17.812493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-775387d2-2292-4b76-88bd-385566e73c95 tempest-ServerRescueNegativeTestJSON-2032582913 tempest-ServerRescueNegativeTestJSON-2032582913-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:17.854973 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Pushing event deleted for resources: {'Port': ['ID=16b2c21e-cb05-472a-b273-fe85f07e77cc,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:17.857602 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:17.949695 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.126696 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.129084 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.316374 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.317068 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 6c5aae18-4faa-4947-af35-4078d428c2b9 {{(pid=99039) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 12:01:18.328791 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.329921 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] 10.222.0.48 "DELETE /v2.0/routers/6c5aae18-4faa-4947-af35-4078d428c2b9 HTTP/1.1" status: 204 len: 168 time: 1.6483526 May 31 12:01:18.333501 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 37982) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:18.391114 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] Deleting subnet 1b74adba-cf2f-4db5-ad92-27ea39726767 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:01:18.393988 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.444945 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.533722 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.661737 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5becf1e-2d92-46ed-b089-ca8de99f338d req-c9b152b7-fee9-43a1-8ed4-cb4ee3d953b6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:18.663698 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b5becf1e-2d92-46ed-b089-ca8de99f338d req-c9b152b7-fee9-43a1-8ed4-cb4ee3d953b6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0940063 May 31 12:01:18.736768 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b5becf1e-2d92-46ed-b089-ca8de99f338d req-9b4c3007-7a54-4654-8bfd-5df005e36306 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0581582 May 31 12:01:18.759623 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b5af8b49-c5f1-494f-aa5a-3d080497d163 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:01:18.808681 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: b5af8b49-c5f1-494f-aa5a-3d080497d163 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:01:18.809295 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b5af8b49-c5f1-494f-aa5a-3d080497d163 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:01:18.812008 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 16b2c21e-cb05-472a-b273-fe85f07e77cc no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:01:18.848568 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 16b2c21e-cb05-472a-b273-fe85f07e77cc {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:01:18.848929 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 16b2c21e-cb05-472a-b273-fe85f07e77cc not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:01:18.882169 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.887273 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:18.928321 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=1b74adba-cf2f-4db5-ad92-27ea39726767,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:18.931182 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:18.937939 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:18.941000 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-bbe18f25-b568-414b-b81d-3d146ad85ab1 None None] Network 36cce22a-502c-464d-a9ac-50f2bc247e07 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:01:18.941300 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-48bd2d66-b729-4e74-9614-62c6d2aafad1 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/1b74adba-cf2f-4db5-ad92-27ea39726767 HTTP/1.1" status: 204 len: 168 time: 0.6077724 May 31 12:01:18.943876 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38002) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:19.143448 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.175999 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38010) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:19.218033 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] DHCP port deletion for 36cce22a-502c-464d-a9ac-50f2bc247e07 request from dv-cinder-791239-4-iscsi {{(pid=99041) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 31 12:01:19.222070 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Deleting port 5be5e898-142f-455d-ac69-dbbd04f043dc {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:01:19.222500 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.285797 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] Deleting port 5be5e898-142f-455d-ac69-dbbd04f043dc {{(pid=99041) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:01:19.286276 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.305812 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-052f27f0-eb68-4381-92fd-e25820421d2c service neutron] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.1282516 May 31 12:01:19.327710 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] For port 5be5e898-142f-455d-ac69-dbbd04f043dc, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=5be5e898-142f-455d-ac69-dbbd04f043dc,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:19.328061 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.392296 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] For port 5be5e898-142f-455d-ac69-dbbd04f043dc, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=5be5e898-142f-455d-ac69-dbbd04f043dc,segment=NetworkSegment(f84631f1-6292-4e5f-a5dd-68b963cdfc7f),segment_id=f84631f1-6292-4e5f-a5dd-68b963cdfc7f)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:19.392696 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.466172 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Calling delete_port for 5be5e898-142f-455d-ac69-dbbd04f043dc owned by network:dhcp {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:01:19.562032 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] Calling delete_port for 5be5e898-142f-455d-ac69-dbbd04f043dc owned by network:dhcp {{(pid=99041) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:01:19.613585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.618550 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8f864828-e6e4-4009-86b3-30f7203319bd tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:19.618993 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.623415 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:19.662798 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Pushing event deleted for resources: {'Port': ['ID=5be5e898-142f-455d-ac69-dbbd04f043dc,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:19.665398 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:19.668189 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] Pushing event deleted for resources: {'Port': ['ID=5be5e898-142f-455d-ac69-dbbd04f043dc,revision_number=None']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:19.670056 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] No DVR routers for this DVR port 5be5e898-142f-455d-ac69-dbbd04f043dc on host dv-cinder-791239-4-iscsi {{(pid=99041) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:01:19.670674 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] No DVR routers for this DVR port 5be5e898-142f-455d-ac69-dbbd04f043dc on host dv-cinder-791239-4-iscsi {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:01:19.671364 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:19.792981 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.806269 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['36cce22a-502c-464d-a9ac-50f2bc247e07']}} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:19.818615 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f') {'for_net_delete': True} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:19.818910 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'f84631f1-6292-4e5f-a5dd-68b963cdfc7f') {} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:01:19.831395 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3096822'] for segment, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.847405 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.853246 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Releasing vlan 363 on physical network physnet1 to pool {{(pid=99040) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:01:19.853650 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2278144'] for segment, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: Traceback (most recent call last): May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: timer() May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: cb(*args, **kw) May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: return func(*args, **kwargs) May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: self._notify() May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: self.callback(batched_events) May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: event.method(event) May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: return f(self, *a, **k) May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._get(url).json() May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: return self.request(url, 'GET', **kwargs) May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:01:19.867443 dv-cinder-791239-4-iscsi neutron-server[99040]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:01:19.949779 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e61a711-7e42-454b-b16b-981984681ab6 req-d81e29c0-738f-4afa-862e-6ee1b1f88678 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:19.951666 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9e61a711-7e42-454b-b16b-981984681ab6 req-d81e29c0-738f-4afa-862e-6ee1b1f88678 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0852246 May 31 12:01:19.995814 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9e61a711-7e42-454b-b16b-981984681ab6 req-d853626c-70ea-4461-9c80-1f4c5af62bb7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0379672 May 31 12:01:20.007618 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:20.026206 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a3cd3976-d4b4-4130-8964-f192e36f3786 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:20.028449 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:20.036716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Pushing event deleted for resources: {'Network': ['ID=36cce22a-502c-464d-a9ac-50f2bc247e07,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:20.039834 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:20.122701 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] 10.222.0.48 "DELETE /v2.0/networks/36cce22a-502c-464d-a9ac-50f2bc247e07 HTTP/1.1" status: 204 len: 168 time: 1.1776705 May 31 12:01:20.510618 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38028) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:20.558060 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-157714a8-6283-43b7-b273-5d1b566caf64 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=b866e6eab1b04406a7280f91698983ce&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0462089 May 31 12:01:20.560500 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38032) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:20.618917 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5a019f48-6437-4ced-8a08-eb4949eceaf9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:20.644333 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5a019f48-6437-4ced-8a08-eb4949eceaf9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:20.727694 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 5be5e898-142f-455d-ac69-dbbd04f043dc no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:01:20.765026 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 5be5e898-142f-455d-ac69-dbbd04f043dc {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:01:20.765026 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 5be5e898-142f-455d-ac69-dbbd04f043dc not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:01:20.784715 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5a019f48-6437-4ced-8a08-eb4949eceaf9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:20.789750 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7be71acb-a057-4369-ac24-8505d747216a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:20.805363 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5a019f48-6437-4ced-8a08-eb4949eceaf9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c242c4b8-d59a-410d-9edf-57e25eb35fd6,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:20.808300 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5a019f48-6437-4ced-8a08-eb4949eceaf9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:20.811184 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-5a019f48-6437-4ced-8a08-eb4949eceaf9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:20.814822 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-5a019f48-6437-4ced-8a08-eb4949eceaf9 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/c242c4b8-d59a-410d-9edf-57e25eb35fd6 HTTP/1.1" status: 204 len: 168 time: 0.2527163 May 31 12:01:20.877966 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-5e374c4e-ab68-4100-abe5-b8a516cded19 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:20.879648 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-5e374c4e-ab68-4100-abe5-b8a516cded19 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77&tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d HTTP/1.1" status: 200 len: 1096 time: 0.0796623 May 31 12:01:20.905807 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2c69fc5d-cdda-46b7-808e-9705041eab53 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:20.973856 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-35257f24-862d-495d-b765-bdf5261350b9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:20.975313 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-35257f24-862d-495d-b765-bdf5261350b9 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 1096 time: 0.0742273 May 31 12:01:21.110444 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-2ad837a9-2fac-4899-8158-58bb380906aa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:21.111885 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-2ad837a9-2fac-4899-8158-58bb380906aa service neutron] 10.222.0.47 "GET /v2.0/networks?id=19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 200 len: 917 time: 0.1285119 May 31 12:01:21.266442 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-306efadd-31d3-4878-94b0-34693d5bf797 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=24163e07-db8d-4a12-8f62-285a1abb77fa HTTP/1.1" status: 200 len: 997 time: 0.1178935 May 31 12:01:21.315370 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ca9aada-606d-4a45-8884-5fecc19f3557 req-586f9620-73d5-475a-855f-e0575e03976e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:21.316815 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1ca9aada-606d-4a45-8884-5fecc19f3557 req-586f9620-73d5-475a-855f-e0575e03976e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.2040911 May 31 12:01:21.366886 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-64061fcd-87f1-41b8-a7cb-4bd22e0c7eaf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:21.368286 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-64061fcd-87f1-41b8-a7cb-4bd22e0c7eaf service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 200 len: 855 time: 0.0945220 May 31 12:01:21.408540 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1ca9aada-606d-4a45-8884-5fecc19f3557 req-6a77991c-1093-40a0-a19f-da2abe586022 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0851119 May 31 12:01:21.453116 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-bdf1b56c-5ded-449f-b391-df77f269f903 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:21.454695 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-bdf1b56c-5ded-449f-b391-df77f269f903 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=19c09cdd-9420-4946-a105-024d5ea86ec3&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0785608 May 31 12:01:21.586435 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-fdcc3c3e-3359-450d-b629-13bebd98bae5 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1241705 May 31 12:01:21.816795 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc250f83-16dd-40ba-b0b1-66bb3266864e req-e6e7d5d7-d56a-46d8-a6ec-a1000fc4b8e8 service neutron] 10.222.0.47 "GET /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2236946 May 31 12:01:22.660988 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fcf0a42a-5043-40ff-9d94-6e7368758dd4 req-f94958ab-ae34-481b-8dd0-ddce8fb25cd9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:22.662601 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fcf0a42a-5043-40ff-9d94-6e7368758dd4 req-f94958ab-ae34-481b-8dd0-ddce8fb25cd9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.1210570 May 31 12:01:22.745132 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fcf0a42a-5043-40ff-9d94-6e7368758dd4 req-e0c04ceb-a5f4-4d75-aa36-15b8e2244927 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0763597 May 31 12:01:23.346295 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:23.447663 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6fbfb2ff-5485-4710-a1d2-fd8e8063244c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:23.941416 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f915e54-26b0-40ae-84eb-f4fae6b0cb37 req-0fee3c2d-32e3-4c89-a914-d7b77586627b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:23.942893 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4f915e54-26b0-40ae-84eb-f4fae6b0cb37 req-0fee3c2d-32e3-4c89-a914-d7b77586627b tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0796227 May 31 12:01:23.987654 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4f915e54-26b0-40ae-84eb-f4fae6b0cb37 req-c161bed4-c1b9-428d-bd4e-5f2a53f3e7c9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0390220 May 31 12:01:24.630633 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:25.182549 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce2f6ba7-c908-448e-82a1-69a3134f6af8 req-3de3fbfc-0df4-4ad0-8594-f4c35db68984 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:25.183964 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce2f6ba7-c908-448e-82a1-69a3134f6af8 req-3de3fbfc-0df4-4ad0-8594-f4c35db68984 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0838847 May 31 12:01:25.226986 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce2f6ba7-c908-448e-82a1-69a3134f6af8 req-1ab4ed00-f120-4454-b09b-420cd21e2938 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0373566 May 31 12:01:26.105530 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:26.351318 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38126) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:26.429340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbfaf66f-b5db-40f5-9cb0-65c95acabf68 req-8d1a4741-bf77-4844-8c63-bf05480b7a28 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:26.430814 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cbfaf66f-b5db-40f5-9cb0-65c95acabf68 req-8d1a4741-bf77-4844-8c63-bf05480b7a28 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0835609 May 31 12:01:26.466477 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:01:26.476193 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cbfaf66f-b5db-40f5-9cb0-65c95acabf68 req-43c58615-e4ac-47af-bd90-a4a1fdc3a8c7 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0391803 May 31 12:01:26.477220 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:26.496807 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:01:26.502502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Usage tracker for resource:security_group_rule and tenant:d8ceffb3f8754b1eab13890fe0c22471 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:01:26.506443 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:01:26.513732 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:01:26.603699 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:26.693690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:26.700769 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-ef6aede4-c084-45ad-bd9b-ffb1b7219573 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:26.742484 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=d8ceffb3f8754b1eab13890fe0c22471&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3900971 May 31 12:01:26.743243 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7a545081-4088-4a65-8d68-cb1ee388accf,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:26.745268 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38132) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:26.746422 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:26.808085 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:26.833389 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:26.962453 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:26.979842 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3480d48a-a71a-4b7f-93f4-fe29ba8ee4c2 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:26.992346 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7a545081-4088-4a65-8d68-cb1ee388accf,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:27.001493 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/7a545081-4088-4a65-8d68-cb1ee388accf HTTP/1.1" status: 204 len: 168 time: 0.2634201 May 31 12:01:27.016275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a65a9088-ea27-4f7e-a171-e14972d1af9f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:27.678089 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64298796-9334-4d5c-b395-422daab57d5b req-771c3bb4-988a-4af6-87ec-ad3857057bb8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:27.679545 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-64298796-9334-4d5c-b395-422daab57d5b req-771c3bb4-988a-4af6-87ec-ad3857057bb8 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0845118 May 31 12:01:27.723906 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-64298796-9334-4d5c-b395-422daab57d5b req-0c25e29f-77e0-48c6-8976-51111db082d1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0388179 May 31 12:01:28.944432 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4abe6b7-eab2-4388-8606-bccd08bafb9c req-f40755d2-3cc6-41aa-ad12-dd4dcf0601ae tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:28.945987 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d4abe6b7-eab2-4388-8606-bccd08bafb9c req-f40755d2-3cc6-41aa-ad12-dd4dcf0601ae tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.1108844 May 31 12:01:29.000067 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d4abe6b7-eab2-4388-8606-bccd08bafb9c req-fcf6fb39-be6b-4b21-8c65-2f6ab89dbc67 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0478771 May 31 12:01:30.317670 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-907435a7-b673-4aea-ac32-244e775c4e70 req-7a95b021-b745-469f-94bf-bb1b8325af68 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:30.319632 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-907435a7-b673-4aea-ac32-244e775c4e70 req-7a95b021-b745-469f-94bf-bb1b8325af68 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.1861880 May 31 12:01:30.374607 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-907435a7-b673-4aea-ac32-244e775c4e70 req-baea0753-5e37-42b0-b16d-a33ade2cb054 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0486393 May 31 12:01:31.564459 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ab5476e-afd0-4fe5-a150-0e46fe14d379 req-325ee2af-0285-416e-b94c-ad2a34feca3d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:31.566081 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3ab5476e-afd0-4fe5-a150-0e46fe14d379 req-325ee2af-0285-416e-b94c-ad2a34feca3d tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0796993 May 31 12:01:31.623830 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3ab5476e-afd0-4fe5-a150-0e46fe14d379 req-85c6c576-d82b-4597-955e-b4d675dc51d6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0518105 May 31 12:01:32.920371 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9fe4f84c-8250-4e8a-ba1e-c2057953f7d3 req-68b20e55-4dfd-49e1-95e1-eebe053ab981 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:32.922717 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9fe4f84c-8250-4e8a-ba1e-c2057953f7d3 req-68b20e55-4dfd-49e1-95e1-eebe053ab981 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.1007071 May 31 12:01:33.016167 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9fe4f84c-8250-4e8a-ba1e-c2057953f7d3 req-9fab8599-3f92-4b9a-af5c-bc5759c00728 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0464721 May 31 12:01:34.278131 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-831444db-fc30-4606-a5b5-209b0affb1b3 req-12d0f848-6489-4cae-be54-fe6c33a03d82 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:34.279954 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-831444db-fc30-4606-a5b5-209b0affb1b3 req-12d0f848-6489-4cae-be54-fe6c33a03d82 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.1327837 May 31 12:01:34.329798 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-831444db-fc30-4606-a5b5-209b0affb1b3 req-cd60365a-91f4-4192-a760-e04932b5adcd tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0430641 May 31 12:01:35.520235 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-153b80f9-2477-4ce5-807b-ee45e1187b61 req-34d419aa-097b-42e9-baa0-c661d802e810 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:35.521727 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-153b80f9-2477-4ce5-807b-ee45e1187b61 req-34d419aa-097b-42e9-baa0-c661d802e810 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0828755 May 31 12:01:35.566764 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-153b80f9-2477-4ce5-807b-ee45e1187b61 req-52e8bfb1-70fb-4d2c-83b9-a5b266c3bef1 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0391419 May 31 12:01:36.754069 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7417fd92-5b27-4a0a-8195-288871124391 req-b22d5454-c9b8-41e7-8a96-ac3ab3bd9eea tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:36.755424 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7417fd92-5b27-4a0a-8195-288871124391 req-b22d5454-c9b8-41e7-8a96-ac3ab3bd9eea tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0795977 May 31 12:01:36.799234 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7417fd92-5b27-4a0a-8195-288871124391 req-e7045343-3051-4c79-ad4b-acebff18c9b0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0383141 May 31 12:01:38.003509 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29671bd1-f947-4baf-85cf-151dd51fe8b3 req-d5128e21-c487-4524-93de-95ee6f16891e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:38.005008 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-29671bd1-f947-4baf-85cf-151dd51fe8b3 req-d5128e21-c487-4524-93de-95ee6f16891e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0898781 May 31 12:01:38.050527 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-29671bd1-f947-4baf-85cf-151dd51fe8b3 req-08e033ee-2cd9-405c-81ae-321cdaae19bf tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0398214 May 31 12:01:39.243995 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-99243896-be51-47a6-8c3d-73462fee0f1d req-818fcd3b-b541-4e51-8a36-f0fef74a12cc tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:39.245421 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-99243896-be51-47a6-8c3d-73462fee0f1d req-818fcd3b-b541-4e51-8a36-f0fef74a12cc tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 943 time: 0.0840840 May 31 12:01:39.254053 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:01:39.290889 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-99243896-be51-47a6-8c3d-73462fee0f1d req-82617bda-68fa-4723-a226-91aa0ccc227c tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389695 May 31 12:01:39.340366 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:39.393004 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:39.405995 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'port_id': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'mac_address': 'fa:16:3e:6f:6d:78', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 371, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:01:39.459150 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:39.475857 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 24163e07-db8d-4a12-8f62-285a1abb77fa {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:01:39.491053 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:39.715960 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:39.717782 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:39.785372 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=9']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:39.788356 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:39.915441 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['24163e07-db8d-4a12-8f62-285a1abb77fa'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:01:40.479558 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5cd3301f-02c1-4a06-806a-64e1b38c5200 req-77339898-54ba-49f9-8abf-88ecd7456fbb tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:40.481076 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5cd3301f-02c1-4a06-806a-64e1b38c5200 req-77339898-54ba-49f9-8abf-88ecd7456fbb tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 944 time: 0.0809925 May 31 12:01:40.535674 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5cd3301f-02c1-4a06-806a-64e1b38c5200 req-69e4606e-11ba-411b-9cc3-d5368b226798 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0485234 May 31 12:01:41.261456 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:01:41.329958 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 24163e07-db8d-4a12-8f62-285a1abb77fa triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:01:41.330340 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:41.446049 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:41.472038 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:01:41.479327 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-48d120f9-d3a6-4b6a-b5d8-65d494d3d096" -d '{"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-plugged", "status": "completed", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:01:41.487519 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:01:41.522865 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:01:41 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-99981568-35ec-4f2d-92c1-00f332d5209f x-openstack-request-id: req-99981568-35ec-4f2d-92c1-00f332d5209f {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:01:41.523162 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-plugged", "status": "completed", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:01:41.523452 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-99981568-35ec-4f2d-92c1-00f332d5209f {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:01:41.523852 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'code': 200} May 31 12:01:41.635143 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:41.637149 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-c69390d6-5260-479c-8eda-b9f4b35af6a5 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:41.705478 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=10']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:41.717941 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:41.743942 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-baef0397-321c-4dbd-97c8-7395f6a957d6 req-ba7d8742-fe7a-46d7-8751-9422966c16c0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:41.743942 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-baef0397-321c-4dbd-97c8-7395f6a957d6 req-ba7d8742-fe7a-46d7-8751-9422966c16c0 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 946 time: 0.0952451 May 31 12:01:41.789650 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-baef0397-321c-4dbd-97c8-7395f6a957d6 req-5ef6bd19-fc51-430d-a407-972db7fa0439 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389256 May 31 12:01:41.978886 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b3ce65c5-ea73-4a01-9d83-7bbce8b928eb req-3e1a8e43-0ac5-49fd-8f55-d431f7efb213 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:41.980435 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b3ce65c5-ea73-4a01-9d83-7bbce8b928eb req-3e1a8e43-0ac5-49fd-8f55-d431f7efb213 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 1099 time: 0.0747464 May 31 12:01:42.025413 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b3ce65c5-ea73-4a01-9d83-7bbce8b928eb req-f34374d8-ea65-44ad-8ad6-953f70271214 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0387123 May 31 12:01:42.994198 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec03691f-ae20-4aee-8802-0feb110f4ea6 req-e4aef1ac-778e-435b-a746-a45027e98a96 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:43.007264 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ec03691f-ae20-4aee-8802-0feb110f4ea6 req-e4aef1ac-778e-435b-a746-a45027e98a96 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 946 time: 0.0960772 May 31 12:01:43.048897 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ec03691f-ae20-4aee-8802-0feb110f4ea6 req-3733b218-ee2c-4c8d-81ab-8f77e934d504 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0425034 May 31 12:01:44.240450 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-21573dee-b344-4670-8691-d67f45985e5b req-4c3803a3-050f-405a-ae36-e51573d37e9e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:44.241991 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-21573dee-b344-4670-8691-d67f45985e5b req-4c3803a3-050f-405a-ae36-e51573d37e9e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 946 time: 0.0805252 May 31 12:01:44.290958 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-21573dee-b344-4670-8691-d67f45985e5b req-e927b349-5daa-4f24-bea7-4716d6334261 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0430777 May 31 12:01:45.489127 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4bd4faa-bba6-475e-909e-5e8b4f5f32b3 req-21775bca-ab61-468a-9718-baeddfd12d73 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:01:45.490607 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d4bd4faa-bba6-475e-909e-5e8b4f5f32b3 req-21775bca-ab61-468a-9718-baeddfd12d73 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 946 time: 0.0886426 May 31 12:01:45.535655 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d4bd4faa-bba6-475e-909e-5e8b4f5f32b3 req-f22e8be2-f7d0-4dd0-9d1c-875a41b25aa6 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393064 May 31 12:01:50.909728 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9fbd3b03-5713-4067-9dca-b921d0f7d167 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:53.339783 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:53.463998 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dc867a94-a5c6-41b8-9493-39f6e6e5e9ff None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:54.624719 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:55.063413 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38338) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:55.178203 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:55.192379 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:01:55.196353 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Usage tracker for resource:security_group_rule and tenant:5d154094275e4c1abc82f12301b77c5c is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:01:55.199203 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:01:55.205317 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:01:55.290498 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:55.378302 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:55.385351 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e032e94d-ac76-4a16-8d5c-46a50fdab5be admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:55.424510 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d4cfafb5-8d67-47d7-80b2-79b2f4083a4c,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:55.428734 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=5d154094275e4c1abc82f12301b77c5c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3643317 May 31 12:01:55.429684 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:55.431450 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38344) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:01:55.492413 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-292b4e14-bcda-4433-a5bb-5c6de68e6275 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:55.518620 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-292b4e14-bcda-4433-a5bb-5c6de68e6275 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:55.681922 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-292b4e14-bcda-4433-a5bb-5c6de68e6275 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:55.686576 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c64448ac-d6fe-46e9-b30e-5aa58dd935d9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:01:55.697346 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-292b4e14-bcda-4433-a5bb-5c6de68e6275 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d4cfafb5-8d67-47d7-80b2-79b2f4083a4c,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:01:55.702684 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-292b4e14-bcda-4433-a5bb-5c6de68e6275 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:01:55.704587 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-292b4e14-bcda-4433-a5bb-5c6de68e6275 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:01:55.705815 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-292b4e14-bcda-4433-a5bb-5c6de68e6275 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/d4cfafb5-8d67-47d7-80b2-79b2f4083a4c HTTP/1.1" status: 204 len: 168 time: 0.2735081 May 31 12:01:56.171681 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:02.346837 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38412) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:02.462192 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:02.476112 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:02.479931 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Usage tracker for resource:security_group_rule and tenant:73ca7861562246d28d6a5507fe77256d is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:02.483210 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:02.488758 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:02.568942 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:02.652358 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:02.659730 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bb04a5f2-733f-4d71-b076-34d94db7bd24 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:02.706675 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=73ca7861562246d28d6a5507fe77256d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3565967 May 31 12:02:02.707213 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38418) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:02.710678 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=62f7d0ed-bd38-40aa-84d3-9efad155477d,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:02.713773 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e8d22a75-bf65-49a8-b2f3-386fa7be49c0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:02.765665 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:02.791762 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:02.915690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:02.920785 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1e156dd6-62f4-488e-98aa-5275282a6cd0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:02.935419 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=62f7d0ed-bd38-40aa-84d3-9efad155477d,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:02.945211 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.945593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.945962 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.946316 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.946662 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.947013 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.947417 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.947765 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.948107 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.951593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.951593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.951593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:02.953328 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:02.955498 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:02.956494 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-243791e0-82b5-4cc7-ba47-1b886f4bf86e admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/62f7d0ed-bd38-40aa-84d3-9efad155477d HTTP/1.1" status: 204 len: 168 time: 0.2483182 May 31 12:02:03.466390 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:02:08.223488 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38468) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:08.340019 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:08.360442 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:08.364244 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Usage tracker for resource:security_group_rule and tenant:2f56cbe9fab741edae8b2b3931a43ecb is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:08.367061 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:08.373942 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:08.457113 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:08.567242 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:08.574466 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-82c2fae6-f36d-4c38-ba20-9db1bfa61312 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:08.614162 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=cf19b082-c7ed-4b5d-9c95-c5bcea176232,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:08.618463 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=2f56cbe9fab741edae8b2b3931a43ecb&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3938534 May 31 12:02:08.619432 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:08.621037 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38474) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:08.680173 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1d11cf0f-cfe6-44ac-ad74-f3acc544fc48 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:08.705305 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1d11cf0f-cfe6-44ac-ad74-f3acc544fc48 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:08.829861 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1d11cf0f-cfe6-44ac-ad74-f3acc544fc48 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:08.833474 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5d47551d-f708-4c2e-b565-bb1a69787d5b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:08.844635 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1d11cf0f-cfe6-44ac-ad74-f3acc544fc48 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=cf19b082-c7ed-4b5d-9c95-c5bcea176232,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:08.849458 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1d11cf0f-cfe6-44ac-ad74-f3acc544fc48 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:08.851427 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1d11cf0f-cfe6-44ac-ad74-f3acc544fc48 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:08.852529 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1d11cf0f-cfe6-44ac-ad74-f3acc544fc48 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/cf19b082-c7ed-4b5d-9c95-c5bcea176232 HTTP/1.1" status: 204 len: 168 time: 0.2305524 May 31 12:02:09.355789 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:09.900560 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38498) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:10.009435 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Request body: {'network': {'name': 'tempest-VolumesSnapshotTestJSON-2053689373-network', 'tenant_id': 'e0fbc2aa94f84d079cbe91bba91e2516'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:02:10.014950 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:10.018713 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Usage tracker for resource:network and tenant:e0fbc2aa94f84d079cbe91bba91e2516 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:10.021493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:10.027098 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:10.138950 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Made reservation on behalf of e0fbc2aa94f84d079cbe91bba91e2516 for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:02:10.141324 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:10.158039 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:10.174166 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:10.178017 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Usage tracker for resource:security_group_rule and tenant:e0fbc2aa94f84d079cbe91bba91e2516 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:10.180958 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:10.187181 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:10.267392 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:10.367120 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:10.370007 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e833a9aa-95e5-45a1-92ae-510bcbba9dcc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:10.399582 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] vlan segment allocate from pool success with {'vlan_id': 382, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 12:02:10.408138 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=48250270-c82c-483c-a3cc-9d463d413e15,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:10.411593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:10.413994 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:10.414325 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Added segment 865e26f1-d91f-476e-a356-62b51f1cbd7b of type vlan for network ad7523f0-ba37-4688-b7be-179e0e8eae53 May 31 12:02:10.441447 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:10.527199 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:10.528884 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:10.529541 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:10.534441 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8808b9de-d54f-4bc9-b222-5ac61d3606a9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:10.631990 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Pushing event updated for resources: {'Network': ['ID=ad7523f0-ba37-4688-b7be-179e0e8eae53,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:10.634665 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:10.642446 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network ad7523f0-ba37-4688-b7be-179e0e8eae53 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 12:02:10.682307 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.7804847 May 31 12:02:10.685748 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38504) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:10.691123 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Request body: {'subnet': {'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesSnapshotTestJSON-2053689373-subnet', 'tenant_id': 'e0fbc2aa94f84d079cbe91bba91e2516', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:02:10.695849 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:10.699641 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Usage tracker for resource:subnet and tenant:e0fbc2aa94f84d079cbe91bba91e2516 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:10.702549 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:10.709023 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:10.801263 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Made reservation on behalf of e0fbc2aa94f84d079cbe91bba91e2516 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:02:10.803669 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:11.238678 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:11.243087 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:11.312258 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Pushing event updated for resources: {'Subnet': ['ID=e4a37872-4de9-422e-a54c-71186b6f99d7,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:11.316074 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:11.320013 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Checking segment: {'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:02:11.320359 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Network hostable DHCP agents. Network: ad7523f0-ba37-4688-b7be-179e0e8eae53, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 12:02:11.407010 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Network ad7523f0-ba37-4688-b7be-179e0e8eae53 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 12:02:11.504530 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, ad7523f0-ba37-4688-b7be-179e0e8eae53) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:02:11.514047 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4cb74ba6-8720-4aa0-91a7-ea53d7cc79ba None None] Network ad7523f0-ba37-4688-b7be-179e0e8eae53 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:02:11.514233 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:11.514765 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:11.620797 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8e5d39aa-5690-4a77-9821-560c5a76161d admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 0.9266584 May 31 12:02:11.734266 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-8865e1f2-228c-4511-9dad-deae44877941 None None] Network ad7523f0-ba37-4688-b7be-179e0e8eae53 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:02:14.088982 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-028ca80d-84d1-4d8c-9c45-114a9b6e7c3a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:14.089595 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-028ca80d-84d1-4d8c-9c45-114a9b6e7c3a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Synchronizing usage tracker for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:network {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:02:14.182121 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-028ca80d-84d1-4d8c-9c45-114a9b6e7c3a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Unset dirty status for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:network {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:02:14.183278 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-028ca80d-84d1-4d8c-9c45-114a9b6e7c3a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/networks?id=ad7523f0-ba37-4688-b7be-179e0e8eae53 HTTP/1.1" status: 200 len: 814 time: 0.3225570 May 31 12:02:14.198421 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-4e583c0b-e9cd-44cf-ba5e-faab256de2a4 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/quotas/e0fbc2aa94f84d079cbe91bba91e2516 HTTP/1.1" status: 200 len: 375 time: 0.0088735 May 31 12:02:14.247779 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-c5fa6b73-302a-4321-b101-78b57b4022bb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0444672 May 31 12:02:14.426952 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-543cde54-26b9-4f53-8350-45ffc3a9decf service neutron] 10.222.0.48 "GET /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1724453 May 31 12:02:14.558241 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-2adf3669-779f-40ea-a50c-8da5d0dae970 service neutron] 10.222.0.48 "GET /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1255574 May 31 12:02:14.885878 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9013ab16-4dd4-447c-ad5f-7206da4e2db8 req-a529e5f3-bc14-4481-a461-402f5ff9ca3e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 210 time: 0.0467651 May 31 12:02:16.033773 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-97056e0c-5d7f-4a27-951f-c77e8f6af5a7 req-2f7ab83d-06f6-4a23-bb90-83eba3a622b5 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 210 time: 0.0432069 May 31 12:02:17.190710 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-17709308-30b7-4c28-9f67-15132a17fdd3 req-27be2f65-0f1e-4dfe-8dd4-5709c7acfb0d tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 210 time: 0.0484488 May 31 12:02:17.389106 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0dae83aa-bb9a-4a4e-afa9-6f23f5842098 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:17.390543 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0dae83aa-bb9a-4a4e-afa9-6f23f5842098 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.47 "GET /v2.0/networks?id=ad7523f0-ba37-4688-b7be-179e0e8eae53 HTTP/1.1" status: 200 len: 814 time: 0.2903011 May 31 12:02:17.405060 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Request body: {'port': {'device_id': '9c533e31-9cae-413c-a2e1-1be300a4878e', 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53', 'admin_state_up': True, 'tenant_id': 'e0fbc2aa94f84d079cbe91bba91e2516'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:02:17.431269 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4d083178-eace-4be2-a686-a7887286951b req-e6127049-0d3f-4eb3-8658-400e4f7a2e54 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:17.432705 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4d083178-eace-4be2-a686-a7887286951b req-e6127049-0d3f-4eb3-8658-400e4f7a2e54 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 946 time: 0.1457613 May 31 12:02:17.562528 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4d083178-eace-4be2-a686-a7887286951b req-0b94a626-1864-4e59-b306-3d651104f6ab tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1228895 May 31 12:02:17.587783 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:17.592035 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Usage tracker for resource:port and tenant:e0fbc2aa94f84d079cbe91bba91e2516 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:17.594927 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:17.600582 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:17.703938 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Made reservation on behalf of e0fbc2aa94f84d079cbe91bba91e2516 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:02:17.707392 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:17.873423 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Allocated IP 172.20.0.10 (ad7523f0-ba37-4688-b7be-179e0e8eae53/e4a37872-4de9-422e-a54c-71186b6f99d7/a2d3edee-28b8-4d43-9cd4-e905470dbb33) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:02:18.085007 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:18.201858 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:18.207699 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:18.273491 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Network ad7523f0-ba37-4688-b7be-179e0e8eae53 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:02:18.276706 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:02:18.280517 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:02:18.320355 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Pushing event updated for resources: {'Port': ['ID=a2d3edee-28b8-4d43-9cd4-e905470dbb33,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:18.327434 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:18.356628 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:18.357154 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:18.389732 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95599330-2924-4b96-92db-e536ff235728 req-d5840b64-d4fb-47f5-b574-92a00a459a72 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:18.391167 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-95599330-2924-4b96-92db-e536ff235728 req-d5840b64-d4fb-47f5-b574-92a00a459a72 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 932 time: 0.0871522 May 31 12:02:18.436752 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-95599330-2924-4b96-92db-e536ff235728 req-d705c3d0-5633-423e-a1fa-44251d06fc16 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Synchronizing usage tracker for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:security_group {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:02:18.463114 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.0627108 May 31 12:02:18.475094 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Request body: {'port': {'device_id': '9c533e31-9cae-413c-a2e1-1be300a4878e', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:02:18.558491 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-95599330-2924-4b96-92db-e536ff235728 req-d705c3d0-5633-423e-a1fa-44251d06fc16 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Unset dirty status for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:security_group {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:02:18.559307 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-95599330-2924-4b96-92db-e536ff235728 req-d705c3d0-5633-423e-a1fa-44251d06fc16 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1621118 May 31 12:02:18.586307 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:18.603522 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-a7a4744e-0592-4f24-b4a3-12d8fde246be tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:18.605215 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-a7a4744e-0592-4f24-b4a3-12d8fde246be tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 946 time: 0.0828531 May 31 12:02:18.747930 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Deleting port 24163e07-db8d-4a12-8f62-285a1abb77fa {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:02:18.748266 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:18.890661 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] For port 24163e07-db8d-4a12-8f62-285a1abb77fa, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=24163e07-db8d-4a12-8f62-285a1abb77fa,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:02:18.891013 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:18.896301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18414a95-c588-4a1d-a20a-66078b4167ab req-bd5c3096-5478-4573-97a0-efac4b7919af tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:18.897859 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-18414a95-c588-4a1d-a20a-66078b4167ab req-bd5c3096-5478-4573-97a0-efac4b7919af tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 946 time: 0.1881552 May 31 12:02:18.967472 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-18414a95-c588-4a1d-a20a-66078b4167ab req-68007059-2bb8-4029-b10e-b7c8081462ab tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups?id=f3375ede-3e88-45fd-8179-4673610130ba&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0588691 May 31 12:02:19.037495 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id a2d3edee-28b8-4d43-9cd4-e905470dbb33 {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:02:19.045690 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:19.060585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Transition to ACTIVE for port object a2d3edee-28b8-4d43-9cd4-e905470dbb33 will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:02:19.149786 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:19.155648 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:19.182525 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:19.182894 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:19.183229 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.l3_db [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Floating IP e6d08467-4c4b-4519-a294-dd4711658a52 disassociated. External IP: 172.27.1.9, port: 24163e07-db8d-4a12-8f62-285a1abb77fa. May 31 12:02:19.183560 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Calling delete_port for 24163e07-db8d-4a12-8f62-285a1abb77fa owned by compute:nova {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:02:19.283610 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Pushing event updated for resources: {'Port': ['ID=a2d3edee-28b8-4d43-9cd4-e905470dbb33,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:19.310027 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:19.359846 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:19.366891 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d8aa8477-4b03-4b37-a179-a39067d8d6a0 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:19.385221 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Attempting to bind port a2d3edee-28b8-4d43-9cd4-e905470dbb33 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:02:19.385546 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Attempting to bind port a2d3edee-28b8-4d43-9cd4-e905470dbb33 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:02:19.385893 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Attempting to bind port a2d3edee-28b8-4d43-9cd4-e905470dbb33 on network ad7523f0-ba37-4688-b7be-179e0e8eae53 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:02:19.391540 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Port a2d3edee-28b8-4d43-9cd4-e905470dbb33 on network ad7523f0-ba37-4688-b7be-179e0e8eae53 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 12:02:19.391905 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Attempting to bind port a2d3edee-28b8-4d43-9cd4-e905470dbb33 on network ad7523f0-ba37-4688-b7be-179e0e8eae53 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:02:19.396892 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 2, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:02:19.398724 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Checking segment: {'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:02:19.398724 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Bound using segment: {'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:02:19.398724 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Bound port: a2d3edee-28b8-4d43-9cd4-e905470dbb33, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:02:19.398724 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:19.417066 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event deleted for resources: {'Port': ['ID=24163e07-db8d-4a12-8f62-285a1abb77fa,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:19.420089 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:19.597033 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] For port a2d3edee-28b8-4d43-9cd4-e905470dbb33, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:02:19.600395 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] For port a2d3edee-28b8-4d43-9cd4-e905470dbb33, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a2d3edee-28b8-4d43-9cd4-e905470dbb33,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:02:19.691624 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:19.695401 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3b591a86-9202-497c-ad05-2b14943936ca tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:19.759667 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:19.760134 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:19.761495 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] 10.222.0.47 "PUT /v2.0/ports/a2d3edee-28b8-4d43-9cd4-e905470dbb33 HTTP/1.1" status: 200 len: 1094 time: 1.2904596 May 31 12:02:19.762739 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9c533e31-9cae-413c-a2e1-1be300a4878e', 'tag': 'a2d3edee-28b8-4d43-9cd4-e905470dbb33'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:02:19.765322 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-23cfac5d-7429-44a6-9a8c-ee5c9375325b" -d '{"events": [{"name": "network-changed", "server_uuid": "9c533e31-9cae-413c-a2e1-1be300a4878e", "tag": "a2d3edee-28b8-4d43-9cd4-e905470dbb33"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:02:19.773593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:02:19.789230 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Pushing event updated for resources: {'Port': ['ID=a2d3edee-28b8-4d43-9cd4-e905470dbb33,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:19.799182 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:19.799702 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-90f472f8-9f5d-4a88-b8da-dea2398c8ccd req-389d8f7d-48a9-4469-988f-33d24304a1c0 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:19.801263 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90f472f8-9f5d-4a88-b8da-dea2398c8ccd req-389d8f7d-48a9-4469-988f-33d24304a1c0 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 944 time: 0.1278632 May 31 12:02:19.815340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:02:19 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-905d4d03-1e19-457e-8663-8c2e0ee62392 x-openstack-request-id: req-905d4d03-1e19-457e-8663-8c2e0ee62392 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:02:19.815604 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "9c533e31-9cae-413c-a2e1-1be300a4878e", "tag": "a2d3edee-28b8-4d43-9cd4-e905470dbb33", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:02:19.815884 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-905d4d03-1e19-457e-8663-8c2e0ee62392 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:02:19.816304 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9c533e31-9cae-413c-a2e1-1be300a4878e', 'tag': 'a2d3edee-28b8-4d43-9cd4-e905470dbb33', 'status': 'completed', 'code': 200} May 31 12:02:19.913618 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-23cfac5d-7429-44a6-9a8c-ee5c9375325b req-a8eae5e9-977c-43b1-acc9-304fb2eb48fa service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:19.915220 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-23cfac5d-7429-44a6-9a8c-ee5c9375325b req-a8eae5e9-977c-43b1-acc9-304fb2eb48fa service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 1097 time: 0.0775633 May 31 12:02:19.975984 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-90f472f8-9f5d-4a88-b8da-dea2398c8ccd req-6d93523a-be7a-47a8-80a0-386ec72a2fdb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1215200 May 31 12:02:20.022879 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:02:20.033478 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] Sync routers for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:02:20.044351 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:02:20.045281 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:01:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:20.078592 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-23cfac5d-7429-44a6-9a8c-ee5c9375325b req-406cfa77-805d-411c-ac60-880a9fc7ca4b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:20.080373 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-23cfac5d-7429-44a6-9a8c-ee5c9375325b req-406cfa77-805d-411c-ac60-880a9fc7ca4b service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&shared=False HTTP/1.1" status: 200 len: 915 time: 0.1580369 May 31 12:02:20.106863 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] No DVR routers for this DVR port 24163e07-db8d-4a12-8f62-285a1abb77fa on host hv-cinder-79123 {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:02:20.109486 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:20.112268 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-deleted', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:02:20.114619 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-39451e7d-0ccf-4cbf-ade1-66acfc09b458" -d '{"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-deleted", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:02:20.151801 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-904868b0-f9e2-493b-b499-7293943540b7 req-b983f610-ce18-412d-98e0-292893c0b553 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/ports?device_id=b42c6e19-ae39-4934-8c0f-ef69a5153f77 HTTP/1.1" status: 200 len: 210 time: 0.0667679 May 31 12:02:20.162619 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:02:20 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-426597ef-d9f5-4b0b-9748-a53f0df3549d x-openstack-request-id: req-426597ef-d9f5-4b0b-9748-a53f0df3549d {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:02:20.165453 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "b42c6e19-ae39-4934-8c0f-ef69a5153f77", "name": "network-vif-deleted", "tag": "24163e07-db8d-4a12-8f62-285a1abb77fa", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:02:20.165453 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-426597ef-d9f5-4b0b-9748-a53f0df3549d {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:02:20.165453 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b42c6e19-ae39-4934-8c0f-ef69a5153f77', 'name': 'network-vif-deleted', 'tag': '24163e07-db8d-4a12-8f62-285a1abb77fa', 'status': 'completed', 'code': 200} May 31 12:02:20.225059 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6d7e247b-8a7c-4307-b2b1-2c59386663ee req-b0f0b281-dae9-4bfd-8f2d-7fc1d8d00a5a tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.47 "DELETE /v2.0/ports/24163e07-db8d-4a12-8f62-285a1abb77fa HTTP/1.1" status: 204 len: 173 time: 1.6144607 May 31 12:02:20.258352 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:02:20.269494 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-23cfac5d-7429-44a6-9a8c-ee5c9375325b req-5441952b-8843-472c-8934-75240896e5f5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:20.270977 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-23cfac5d-7429-44a6-9a8c-ee5c9375325b req-5441952b-8843-472c-8934-75240896e5f5 service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1543696 May 31 12:02:20.380438 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-cc3a7b9d-e721-4520-9c1e-b41313371990 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:20.381926 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-cc3a7b9d-e721-4520-9c1e-b41313371990 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 1097 time: 0.0764580 May 31 12:02:20.456737 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-0873a246-1fbf-4753-8279-1539ae056cc6 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=a2d3edee-28b8-4d43-9cd4-e905470dbb33 HTTP/1.1" status: 200 len: 217 time: 0.0675995 May 31 12:02:20.522365 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-61f93618-a4de-49a1-822c-a9d83889d642 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:20.523714 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-61f93618-a4de-49a1-822c-a9d83889d642 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=e4a37872-4de9-422e-a54c-71186b6f99d7 HTTP/1.1" status: 200 len: 854 time: 0.0614636 May 31 12:02:20.572459 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-21f8a3f0-da19-4a75-9af4-18ba2c09a857 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=ad7523f0-ba37-4688-b7be-179e0e8eae53&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0425856 May 31 12:02:20.575577 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:20.579550 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] Checking router: 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:02:20.588658 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:02:20.589259 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:01:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:20.706823 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-6412b2c9-a12d-4e37-a8bd-d62732842a04 service neutron] 10.222.0.47 "GET /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1282794 May 31 12:02:20.821290 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:02:20.854431 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3878ca93-81d8-41f5-a313-857bc8a5d644 req-d9fc01b9-8d50-404d-bb20-461bf1269c08 service neutron] 10.222.0.47 "GET /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1417851 May 31 12:02:20.909671 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-def236e9-d1a5-4f39-86fb-c1ebd7c2eb54 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:21.142398 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:21.146538 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f7e9361e-3364-47c0-8e94-4f06cdb81fb6 None None] The sync data for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi is: [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T11:54:32Z', 'revision_number': 4, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '502e1922-1254-4eed-bdc8-4c83694840b3', 'name': '', 'network_id': '19c09cdd-9420-4946-a105-024d5ea86ec3', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'mac_address': 'fa:16:3e:7c:c6:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:27Z', 'updated_at': '2021-05-31T11:54:47Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'subnets': [{'id': '4c55d678-c52f-4b1e-b005-86edad0fbf12', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:02:21.179392 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c217d112-f76e-4d64-a2ce-6bb33b2370ea req-a9b6dcb5-b7b1-46c4-9450-74bce73d0a57 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:21.180915 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c217d112-f76e-4d64-a2ce-6bb33b2370ea req-a9b6dcb5-b7b1-46c4-9450-74bce73d0a57 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 944 time: 0.0876915 May 31 12:02:21.228346 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c217d112-f76e-4d64-a2ce-6bb33b2370ea req-b388e4df-6398-4539-9bde-5cf5ef7b4f49 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0413139 May 31 12:02:21.352909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38582) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:21.363750 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7c5a2da9-d0d0-40c4-8807-8fc4c785c6a2 None None] New status for floating IP e6d08467-4c4b-4519-a294-dd4711658a52: DOWN {{(pid=99041) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:02:21.418768 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-4ae58cfe-4141-400a-8403-292de5078d22 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:21.446259 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-4ae58cfe-4141-400a-8403-292de5078d22 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:21.491018 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:02:21.544275 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 24163e07-db8d-4a12-8f62-285a1abb77fa {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:02:21.544626 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 24163e07-db8d-4a12-8f62-285a1abb77fa not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:02:21.570920 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-4ae58cfe-4141-400a-8403-292de5078d22 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:21.576123 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0e916296-54d4-47e4-88b9-3e14fec4f8d1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:21.591506 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4ae58cfe-4141-400a-8403-292de5078d22 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=f3375ede-3e88-45fd-8179-4673610130ba,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:21.593937 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-4ae58cfe-4141-400a-8403-292de5078d22 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:21.658521 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-4ae58cfe-4141-400a-8403-292de5078d22 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:21.659621 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-4ae58cfe-4141-400a-8403-292de5078d22 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "DELETE /v2.0/security-groups/f3375ede-3e88-45fd-8179-4673610130ba HTTP/1.1" status: 204 len: 168 time: 0.3087611 May 31 12:02:21.662351 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38586) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:21.681752 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.api.v2.resource [None req-03973cf4-5828-4fde-b4ef-e8a006127a05 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] show failed (client error): The resource could not be found. May 31 12:02:21.682603 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-03973cf4-5828-4fde-b4ef-e8a006127a05 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "GET /v2.0/security-groups/f3375ede-3e88-45fd-8179-4673610130ba HTTP/1.1" status: 404 len: 347 time: 0.0193994 May 31 12:02:21.685507 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38590) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:21.901884 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Deleting port 56020899-a5d2-4887-8fc3-6fe9b3416c61 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:02:21.902605 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:22.016810 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:22.017050 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:22.232988 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Calling delete_port for 56020899-a5d2-4887-8fc3-6fe9b3416c61 owned by network:floatingip {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:02:22.420379 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2a2e437c-c988-4f9d-9145-d9cfcd9b6cce req-7b51616c-92aa-4feb-953c-1c74ce45ac7e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:22.421067 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-2a2e437c-c988-4f9d-9145-d9cfcd9b6cce req-7b51616c-92aa-4feb-953c-1c74ce45ac7e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Synchronizing usage tracker for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:02:22.441209 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:22.445882 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-41817f88-a20c-4a8c-9377-45bb52e1c2a0 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:22.517215 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Pushing event deleted for resources: {'Port': ['ID=56020899-a5d2-4887-8fc3-6fe9b3416c61,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:22.519703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:22.557056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-2a2e437c-c988-4f9d-9145-d9cfcd9b6cce req-7b51616c-92aa-4feb-953c-1c74ce45ac7e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Unset dirty status for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:02:22.558229 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2a2e437c-c988-4f9d-9145-d9cfcd9b6cce req-7b51616c-92aa-4feb-953c-1c74ce45ac7e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 944 time: 0.2176993 May 31 12:02:22.611443 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks [] for floatingip, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:22.624149 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2a2e437c-c988-4f9d-9145-d9cfcd9b6cce req-1fbbd815-7a48-47b7-9c7d-ea0c5ab00f34 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0587661 May 31 12:02:22.631806 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:22.632839 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] 10.222.0.48 "DELETE /v2.0/floatingips/e6d08467-4c4b-4519-a294-dd4711658a52 HTTP/1.1" status: 204 len: 168 time: 0.9463420 May 31 12:02:23.395325 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:23.479290 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device a2d3edee-28b8-4d43-9cd4-e905470dbb33 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:02:23.556160 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dd4446ea-49d1-4508-8aad-3412b986d254 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:23.615186 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port a2d3edee-28b8-4d43-9cd4-e905470dbb33, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a2d3edee-28b8-4d43-9cd4-e905470dbb33,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:02:23.628558 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'a2d3edee-28b8-4d43-9cd4-e905470dbb33', 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53', 'port_id': 'a2d3edee-28b8-4d43-9cd4-e905470dbb33', 'mac_address': 'fa:16:3e:d3:f3:35', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 382, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'e4a37872-4de9-422e-a54c-71186b6f99d7', 'ip_address': '172.20.0.10'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:02:23.688192 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:23.705263 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id a2d3edee-28b8-4d43-9cd4-e905470dbb33 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:02:23.720610 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port a2d3edee-28b8-4d43-9cd4-e905470dbb33, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a2d3edee-28b8-4d43-9cd4-e905470dbb33,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:02:23.807975 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c40cf14-0e85-430c-a810-e8b20661c5a3 req-b4206d8e-331a-4e52-9c42-fb37c6306861 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:23.810002 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9c40cf14-0e85-430c-a810-e8b20661c5a3 req-b4206d8e-331a-4e52-9c42-fb37c6306861 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 944 time: 0.0802262 May 31 12:02:23.824245 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:23.826954 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:23.854393 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9c40cf14-0e85-430c-a810-e8b20661c5a3 req-81191f47-62b8-42ce-b27d-cbe8e3148843 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0388954 May 31 12:02:23.893316 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=a2d3edee-28b8-4d43-9cd4-e905470dbb33,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:23.900362 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.900701 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.901071 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.901400 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.901757 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.902081 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.902500 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.903020 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.903379 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.903700 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.904018 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.904336 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:23.908075 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:24.042053 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['a2d3edee-28b8-4d43-9cd4-e905470dbb33'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:02:24.612312 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:25.042433 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-48a12881-82bd-4711-821e-65784ca993cb req-c4bd98e3-fa23-4326-9063-57123eae137d tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:25.044332 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-48a12881-82bd-4711-821e-65784ca993cb req-c4bd98e3-fa23-4326-9063-57123eae137d tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 945 time: 0.0856936 May 31 12:02:25.088472 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-48a12881-82bd-4711-821e-65784ca993cb req-cb047aff-55e5-4016-a73f-4050f0a5353d tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384128 May 31 12:02:25.479440 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device a2d3edee-28b8-4d43-9cd4-e905470dbb33 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:02:25.601282 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port a2d3edee-28b8-4d43-9cd4-e905470dbb33 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:02:25.607593 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port a2d3edee-28b8-4d43-9cd4-e905470dbb33 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:02:25.607975 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:25.694520 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38622) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:25.734375 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:25.762054 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9c533e31-9cae-413c-a2e1-1be300a4878e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a2d3edee-28b8-4d43-9cd4-e905470dbb33'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:02:25.764547 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-4ab4ca0c-1e6d-4197-a07b-c094bc981005" -d '{"events": [{"server_uuid": "9c533e31-9cae-413c-a2e1-1be300a4878e", "name": "network-vif-plugged", "status": "completed", "tag": "a2d3edee-28b8-4d43-9cd4-e905470dbb33"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:02:25.772140 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port a2d3edee-28b8-4d43-9cd4-e905470dbb33, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a2d3edee-28b8-4d43-9cd4-e905470dbb33,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:02:25.808009 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:02:25 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-a2a1d2ba-5261-4cc1-bbbb-c5a890d9d2b2 x-openstack-request-id: req-a2a1d2ba-5261-4cc1-bbbb-c5a890d9d2b2 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:02:25.808302 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "9c533e31-9cae-413c-a2e1-1be300a4878e", "name": "network-vif-plugged", "status": "completed", "tag": "a2d3edee-28b8-4d43-9cd4-e905470dbb33", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:02:25.808631 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-a2a1d2ba-5261-4cc1-bbbb-c5a890d9d2b2 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:02:25.809724 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9c533e31-9cae-413c-a2e1-1be300a4878e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a2d3edee-28b8-4d43-9cd4-e905470dbb33', 'code': 200} May 31 12:02:25.869577 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:25.871989 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-3fcfba53-4afe-487e-8817-0bbea2a5e6ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:25.939495 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=a2d3edee-28b8-4d43-9cd4-e905470dbb33,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:25.942578 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:26.115797 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:26.286500 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:26.343437 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-409aeb99-18bb-4e55-9949-2dbae694e1f3 req-2a68a8ec-a1d3-4ab2-9134-0e5f8f295b33 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:26.345017 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-409aeb99-18bb-4e55-9949-2dbae694e1f3 req-2a68a8ec-a1d3-4ab2-9134-0e5f8f295b33 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 1099 time: 0.1365469 May 31 12:02:26.352471 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0797dca1-7e45-49a0-b4c5-17440eaa26ee req-51a6bd42-cd4c-4d76-a6c7-b517bae09fa8 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:26.353821 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0797dca1-7e45-49a0-b4c5-17440eaa26ee req-51a6bd42-cd4c-4d76-a6c7-b517bae09fa8 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 946 time: 0.1474211 May 31 12:02:26.419554 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-409aeb99-18bb-4e55-9949-2dbae694e1f3 req-20ed3b5e-3476-40c0-ba84-0021987447a6 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0647838 May 31 12:02:26.424192 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0797dca1-7e45-49a0-b4c5-17440eaa26ee req-1b2c010b-366a-437e-ad81-af505e6fd99a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0567675 May 31 12:02:26.825955 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Deleting port 502e1922-1254-4eed-bdc8-4c83694840b3 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:02:26.826562 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:26.945188 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] For port 502e1922-1254-4eed-bdc8-4c83694840b3, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=502e1922-1254-4eed-bdc8-4c83694840b3,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:02:26.945535 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:27.074800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Calling delete_port for 502e1922-1254-4eed-bdc8-4c83694840b3 owned by network:router_interface {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:02:27.302633 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:27.307379 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5fc0788e-b872-4f70-adc4-bb33fddc7610 tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:27.353294 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Pushing event deleted for resources: {'Port': ['ID=502e1922-1254-4eed-bdc8-4c83694840b3,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:27.379330 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:27.675417 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:27.782123 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2147d6a9-1250-4b99-beb2-d2ec1fbdcb8d req-cc784349-6a5c-41f0-b0cc-eff7b2fca3be tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:27.783579 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2147d6a9-1250-4b99-beb2-d2ec1fbdcb8d req-cc784349-6a5c-41f0-b0cc-eff7b2fca3be tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 946 time: 0.1945941 May 31 12:02:27.909847 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2147d6a9-1250-4b99-beb2-d2ec1fbdcb8d req-d32f442a-208d-459f-ae1e-5fc522a1a869 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0714059 May 31 12:02:28.210492 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:02:28.384583 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:02:28.393528 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] Sync routers for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:02:28.397266 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] 10.222.0.48 "PUT /v2.0/routers/8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.7017155 May 31 12:02:28.399781 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38672) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:28.403782 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:02:28.405929 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:02:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:28.583288 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks [] for router, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:28.592376 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:02:28.761416 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:28.889086 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185'}], 'enable_snat': True}, 'gw_port_id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T12:02:27Z', 'revision_number': 5, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port': {'id': 'b2b561e9-692d-4b72-911d-dd2b5feeee1c', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:48:c1:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 87, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:54:18Z', 'updated_at': '2021-05-31T11:54:34Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:28.893242 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] Checking router: 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:02:28.902936 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] Routers {'8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:02:28.903242 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:02:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:28.950130 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Deleting port b2b561e9-692d-4b72-911d-dd2b5feeee1c {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:02:28.950665 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:29.054187 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] Set router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 gateway port host: None {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:02:29.080652 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] For port b2b561e9-692d-4b72-911d-dd2b5feeee1c, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b2b561e9-692d-4b72-911d-dd2b5feeee1c,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:02:29.081007 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:29.160379 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T12:02:28Z', 'revision_number': 6, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port_host': None}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:29.164402 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7d9ace72-9174-4d55-bcfe-af90903bfc29 None None] The sync data for ids ['8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6'] in dv-cinder-791239-4-iscsi is: [{'id': '8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6', 'name': 'tempest-AttachVolumeShelveTestJSON-831574982-router', 'tenant_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:54:16Z', 'updated_at': '2021-05-31T12:02:28Z', 'revision_number': 6, 'project_id': '0e7f0a333ee54b31bcd5c2290c3e284d', 'gw_port_host': None}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:02:29.210272 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Calling delete_port for b2b561e9-692d-4b72-911d-dd2b5feeee1c owned by network:router_gateway {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:02:29.435772 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:29.439689 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8ca2b517-e127-4b09-a8aa-03080f9748cd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:29.511272 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Pushing event deleted for resources: {'Port': ['ID=b2b561e9-692d-4b72-911d-dd2b5feeee1c,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:29.519087 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:29.585489 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:29.761674 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:29.764610 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:29.995455 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:29.996206 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Fanout notify agent at l3_agent the message router_deleted on router 8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 {{(pid=99039) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 12:02:30.009727 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.011190 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-31d4906f-11d9-4196-98d6-0e9f493a834e admin admin] 10.222.0.48 "DELETE /v2.0/routers/8ed55127-0e26-4ec1-bcc1-b82f8cf4e4b6 HTTP/1.1" status: 204 len: 168 time: 1.6101909 May 31 12:02:30.015041 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38696) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:30.081063 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_v2 [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Deleting subnet 4c55d678-c52f-4b1e-b005-86edad0fbf12 {{(pid=99040) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:02:30.083926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.133351 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3096829'] for subnet, precommit_delete_associations {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.220761 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.538633 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2111654', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2831123'] for subnet, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.542784 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5927bbfd-b6c0-48d8-b51f-a7df710088e0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:30.581119 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=4c55d678-c52f-4b1e-b005-86edad0fbf12,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:30.586230 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.591391 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-76a6958d-6785-4362-8f9d-96c5e83e9e0a None None] Network 19c09cdd-9420-4946-a105-024d5ea86ec3 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:02:30.593745 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.594119 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.594475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.594824 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.595170 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.595513 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.595853 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.596190 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.596529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.596864 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.597199 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.597532 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:02:30.602400 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/4c55d678-c52f-4b1e-b005-86edad0fbf12 HTTP/1.1" status: 204 len: 168 time: 0.5857110 May 31 12:02:30.603369 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c86d6db5-7ec1-4603-9120-32e2eff32887 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:30.605012 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38700) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:30.716531 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.785834 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 502e1922-1254-4eed-bdc8-4c83694840b3 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:02:30.790713 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Deleting port 02e88ea0-e503-4523-95a4-11716e4aff1b {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:02:30.791053 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.857847 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 502e1922-1254-4eed-bdc8-4c83694840b3 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:02:30.858195 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 502e1922-1254-4eed-bdc8-4c83694840b3 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:02:30.858602 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b2b561e9-692d-4b72-911d-dd2b5feeee1c no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:02:30.894496 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: b2b561e9-692d-4b72-911d-dd2b5feeee1c {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:02:30.894833 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b2b561e9-692d-4b72-911d-dd2b5feeee1c not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:02:30.899354 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] For port 02e88ea0-e503-4523-95a4-11716e4aff1b, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=02e88ea0-e503-4523-95a4-11716e4aff1b,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:02:30.899726 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:30.961965 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] DHCP port deletion for 19c09cdd-9420-4946-a105-024d5ea86ec3 request from dv-cinder-791239-4-iscsi {{(pid=99041) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 31 12:02:31.029617 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Calling delete_port for 02e88ea0-e503-4523-95a4-11716e4aff1b owned by network:dhcp {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:02:31.030220 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] Deleting port 02e88ea0-e503-4523-95a4-11716e4aff1b {{(pid=99041) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:02:31.030220 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.127949 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] For port 02e88ea0-e503-4523-95a4-11716e4aff1b, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=02e88ea0-e503-4523-95a4-11716e4aff1b,segment=NetworkSegment(15b69bd1-9fea-43fa-badd-39be1ac15eb5),segment_id=15b69bd1-9fea-43fa-badd-39be1ac15eb5)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:02:31.128337 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.230438 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.234308 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-95f49b71-68c9-4cc3-9031-574f5b0007d5 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:31.251103 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] Calling delete_port for 02e88ea0-e503-4523-95a4-11716e4aff1b owned by network:dhcp {{(pid=99041) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:02:31.283193 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Pushing event deleted for resources: {'Port': ['ID=02e88ea0-e503-4523-95a4-11716e4aff1b,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:31.286352 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:31.287121 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] No DVR routers for this DVR port 02e88ea0-e503-4523-95a4-11716e4aff1b on host dv-cinder-791239-4-iscsi {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:02:31.293878 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.297371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:31.340380 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] Pushing event deleted for resources: {'Port': ['ID=02e88ea0-e503-4523-95a4-11716e4aff1b,revision_number=None']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:31.343691 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:31.345985 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] No DVR routers for this DVR port 02e88ea0-e503-4523-95a4-11716e4aff1b on host dv-cinder-791239-4-iscsi {{(pid=99041) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:02:31.415588 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.429195 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['19c09cdd-9420-4946-a105-024d5ea86ec3']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:31.441850 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '15b69bd1-9fea-43fa-badd-39be1ac15eb5') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:31.442184 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '15b69bd1-9fea-43fa-badd-39be1ac15eb5') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:31.453551 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.470410 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.476409 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Releasing vlan 371 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:02:31.476792 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:02:31.490886 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:02:31.650644 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.654917 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:31.668304 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:31.678758 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Pushing event deleted for resources: {'Network': ['ID=19c09cdd-9420-4946-a105-024d5ea86ec3,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:31.682192 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:31.756111 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-de1885ea-a420-4bb4-9efb-687950167408 admin admin] 10.222.0.48 "DELETE /v2.0/networks/19c09cdd-9420-4946-a105-024d5ea86ec3 HTTP/1.1" status: 204 len: 168 time: 1.1497834 May 31 12:02:32.097115 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38710) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:32.146841 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-990129b6-91f0-4f43-aadc-1539e7f97805 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=0e7f0a333ee54b31bcd5c2290c3e284d&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0481391 May 31 12:02:32.149023 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38714) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:32.215812 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f2251397-2ae6-4d67-9de9-523b04ed9a05 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:32.243501 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f2251397-2ae6-4d67-9de9-523b04ed9a05 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:32.378111 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f2251397-2ae6-4d67-9de9-523b04ed9a05 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:32.383735 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bfcec9bf-90cf-4c80-b493-7927ae245f7b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:32.397816 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f2251397-2ae6-4d67-9de9-523b04ed9a05 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f149b027-6e58-4aae-b1ae-31001ccb1da9,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:32.401220 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f2251397-2ae6-4d67-9de9-523b04ed9a05 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:32.404977 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f2251397-2ae6-4d67-9de9-523b04ed9a05 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:32.406109 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f2251397-2ae6-4d67-9de9-523b04ed9a05 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/f149b027-6e58-4aae-b1ae-31001ccb1da9 HTTP/1.1" status: 204 len: 168 time: 0.2560749 May 31 12:02:32.774986 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 02e88ea0-e503-4523-95a4-11716e4aff1b no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:02:32.809640 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 02e88ea0-e503-4523-95a4-11716e4aff1b {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:02:32.809994 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 02e88ea0-e503-4523-95a4-11716e4aff1b not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:02:35.175888 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38770) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:35.293339 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:35.307923 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:35.312383 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Usage tracker for resource:security_group_rule and tenant:38698d88f45846e7a5ea9004d1c2b033 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:35.315222 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:35.320940 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:35.405934 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:35.618193 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:35.625691 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d0060474-efc6-423b-93b8-df1580c568d1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:35.667118 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=76264069-6304-44ad-9bd2-4a28b4806dcf,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:35.670686 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:35.673206 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-085a39cb-16b4-45e2-839a-3d149cc00c54 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=38698d88f45846e7a5ea9004d1c2b033&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4963291 May 31 12:02:35.677295 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38778) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:35.737689 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1eed6b99-ca04-44b8-a4cb-167f23c7c172 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:35.769470 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1eed6b99-ca04-44b8-a4cb-167f23c7c172 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:35.904208 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1eed6b99-ca04-44b8-a4cb-167f23c7c172 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:35.908093 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a60dcce3-cfb7-4bf3-8c4e-c44476a7cc88 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:35.921010 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1eed6b99-ca04-44b8-a4cb-167f23c7c172 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=76264069-6304-44ad-9bd2-4a28b4806dcf,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:35.925443 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1eed6b99-ca04-44b8-a4cb-167f23c7c172 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:35.930057 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1eed6b99-ca04-44b8-a4cb-167f23c7c172 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:35.931213 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1eed6b99-ca04-44b8-a4cb-167f23c7c172 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/76264069-6304-44ad-9bd2-4a28b4806dcf HTTP/1.1" status: 204 len: 168 time: 0.2529166 May 31 12:02:36.572397 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38788) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:36.591673 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:36.608146 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:36.611928 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Usage tracker for resource:security_group_rule and tenant:3afc5d3bcf174520a5bc510fb33593db is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:36.614788 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:36.620545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:36.703851 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:36.772808 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:36.780630 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5519c99b-336b-42be-acd2-2662606a0c27 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:36.821131 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=3afc5d3bcf174520a5bc510fb33593db&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2475681 May 31 12:02:36.822147 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ce6a809b-3cdc-42d3-b652-e74d9d0d76da,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:36.824357 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38792) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:36.825847 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-860cfa87-4ae5-4314-b242-cd46b6032d5c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:36.887288 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1231f953-8cd6-476b-af09-c6caa0ece573 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:36.914970 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1231f953-8cd6-476b-af09-c6caa0ece573 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:37.059260 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1231f953-8cd6-476b-af09-c6caa0ece573 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:37.063042 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f6061044-cc50-4885-8008-8cb95fddc5c8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:37.074730 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1231f953-8cd6-476b-af09-c6caa0ece573 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ce6a809b-3cdc-42d3-b652-e74d9d0d76da,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:37.078309 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1231f953-8cd6-476b-af09-c6caa0ece573 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:37.081678 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1231f953-8cd6-476b-af09-c6caa0ece573 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:37.082802 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1231f953-8cd6-476b-af09-c6caa0ece573 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/ce6a809b-3cdc-42d3-b652-e74d9d0d76da HTTP/1.1" status: 204 len: 168 time: 0.2566457 May 31 12:02:38.297976 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38818) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:38.403706 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-416136790-network', 'tenant_id': 'd26c294be9c24ee4a83c99a700cf051e'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:02:38.408755 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:38.412606 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Usage tracker for resource:network and tenant:d26c294be9c24ee4a83c99a700cf051e is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:38.415493 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:38.421934 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:38.492314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Made reservation on behalf of d26c294be9c24ee4a83c99a700cf051e for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:02:38.494721 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:38.511409 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:38.526050 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:38.529993 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Usage tracker for resource:security_group_rule and tenant:d26c294be9c24ee4a83c99a700cf051e is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:38.532777 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:38.538515 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:38.625628 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:38.961336 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:38.965392 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-4ae58cfe-4141-400a-8403-292de5078d22 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:38.993660 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] vlan segment allocate from pool success with {'vlan_id': 372, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 12:02:39.002083 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=55306038-4d32-415e-8d92-a66a660cafaf,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:39.005118 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:39.009836 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:39.010227 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Added segment 5e68b53f-62bb-410e-82e8-536abd023714 of type vlan for network 25d565bb-afcb-4ac9-b31c-e0771872ad08 May 31 12:02:39.037413 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:39.164759 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:39.167247 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:39.168434 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:39.172058 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:39.296743 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Pushing event updated for resources: {'Network': ['ID=25d565bb-afcb-4ac9-b31c-e0771872ad08,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:39.300256 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:39.306311 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 25d565bb-afcb-4ac9-b31c-e0771872ad08 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 12:02:39.390385 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-bc6f734d-689b-4c74-b6df-cbb7dd27176c admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 897 time: 1.0913012 May 31 12:02:39.393400 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38826) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:39.399737 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Request body: {'subnet': {'network_id': '25d565bb-afcb-4ac9-b31c-e0771872ad08', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-416136790-subnet', 'tenant_id': 'd26c294be9c24ee4a83c99a700cf051e', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:02:39.405270 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:39.409320 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Usage tracker for resource:subnet and tenant:d26c294be9c24ee4a83c99a700cf051e is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:39.413989 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:39.414753 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:39.420431 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:39.504520 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Made reservation on behalf of d26c294be9c24ee4a83c99a700cf051e for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:02:39.506902 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:40.105951 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:40.111682 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-3d0f38ff-8610-484d-a2b8-b5b275c0f723 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:40.182585 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Pushing event updated for resources: {'Subnet': ['ID=cfed87a0-8c41-4494-ae4b-5db44cdf1f92,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:40.187137 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:40.191133 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Checking segment: {'id': '5e68b53f-62bb-410e-82e8-536abd023714', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 372, 'network_id': '25d565bb-afcb-4ac9-b31c-e0771872ad08'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:02:40.191488 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Network hostable DHCP agents. Network: 25d565bb-afcb-4ac9-b31c-e0771872ad08, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 12:02:40.301924 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Network 25d565bb-afcb-4ac9-b31c-e0771872ad08 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 12:02:40.393041 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 25d565bb-afcb-4ac9-b31c-e0771872ad08) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:02:40.402099 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:40.402690 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:40.402950 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ac7d9ad4-7977-4911-9e4c-08298394d21d None None] Network 25d565bb-afcb-4ac9-b31c-e0771872ad08 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:02:40.465332 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:02:40.498979 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-40f9982d-9437-40db-8702-d7ad0c22d36a admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 861 time: 1.1042325 May 31 12:02:40.600289 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-048ebac6-49b8-4cf8-ba11-3ac416059070 None None] Network 25d565bb-afcb-4ac9-b31c-e0771872ad08 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:02:41.233540 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38860) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:41.239098 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-2013312260-network', 'tenant_id': '2880850360b744be86bee375846ab5cd'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:02:41.243819 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:41.248168 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Usage tracker for resource:network and tenant:2880850360b744be86bee375846ab5cd is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:41.251019 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:41.256692 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:41.352870 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Made reservation on behalf of 2880850360b744be86bee375846ab5cd for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:02:41.355742 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:41.367908 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:41.382526 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-b6621ba7-fdeb-4fc0-a575-12b623e654d5 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0728178 May 31 12:02:41.386558 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:41.390788 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Usage tracker for resource:security_group_rule and tenant:2880850360b744be86bee375846ab5cd is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:41.393699 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:41.399468 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:41.483425 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:41.522917 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-29442f3e-74c3-46ff-9168-bc017d564486 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:41.524379 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-29442f3e-74c3-46ff-9168-bc017d564486 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1098 time: 0.0723159 May 31 12:02:41.560893 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:41.565123 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:41.591850 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] vlan segment allocate from pool success with {'vlan_id': 357, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 12:02:41.600157 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6e0bab21-9393-432d-bdfd-ef82101235c0,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:41.603714 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:41.606887 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:41.607222 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Added segment f009e1a2-b422-43cd-a0e1-da02b6b83476 of type vlan for network b17682f1-2ab2-4f30-acf6-e88f538ecac2 May 31 12:02:41.633375 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:41.672547 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-86212f1b-9500-43ca-b92a-0623be942262 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:41.674033 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-86212f1b-9500-43ca-b92a-0623be942262 service neutron] 10.222.0.47 "GET /v2.0/networks?id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 200 len: 910 time: 0.1295817 May 31 12:02:41.721207 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:41.722661 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:41.723360 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:41.728306 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:41.836332 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Pushing event updated for resources: {'Network': ['ID=b17682f1-2ab2-4f30-acf6-e88f538ecac2,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:41.838875 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:41.846724 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network b17682f1-2ab2-4f30-acf6-e88f538ecac2 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 12:02:41.873194 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-0c73c304-e983-463a-8a50-b3d3d02e0d61 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 200 len: 1001 time: 0.1920013 May 31 12:02:41.888113 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-04d4e328-1c8a-425a-b3b2-e13934a1d017 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 898 time: 0.6531873 May 31 12:02:41.890708 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38864) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:41.896400 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Request body: {'subnet': {'network_id': 'b17682f1-2ab2-4f30-acf6-e88f538ecac2', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-2013312260-subnet', 'tenant_id': '2880850360b744be86bee375846ab5cd', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:02:41.901619 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:41.905983 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Usage tracker for resource:subnet and tenant:2880850360b744be86bee375846ab5cd is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:41.909181 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:41.915253 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:41.938377 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-7bc152bc-87e7-4296-926d-eedf2dd83cb9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:41.939805 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-7bc152bc-87e7-4296-926d-eedf2dd83cb9 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=3a19e015-3084-4732-bb65-ea124df3118f HTTP/1.1" status: 200 len: 848 time: 0.0590894 May 31 12:02:42.001043 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Made reservation on behalf of 2880850360b744be86bee375846ab5cd for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:02:42.003586 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:42.022518 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-94e4cafc-f4e0-4195-9ce6-7cfc9d722a65 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:42.066066 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-94e4cafc-f4e0-4195-9ce6-7cfc9d722a65 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0769031 May 31 12:02:42.170126 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-ef611ac2-c5c9-4b88-940f-a5b1828a280f service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1382258 May 31 12:02:42.302225 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-014c9c04-a1e1-40da-83dd-75f12de9d85d req-46f955a2-f55f-473e-9764-a59eb541f380 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1252065 May 31 12:02:42.509931 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:42.515603 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:42.583302 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Pushing event updated for resources: {'Subnet': ['ID=a9fe49e6-a14f-4456-a88e-e3d144a0fef9,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:42.587053 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:42.594405 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Checking segment: {'id': 'f009e1a2-b422-43cd-a0e1-da02b6b83476', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 357, 'network_id': 'b17682f1-2ab2-4f30-acf6-e88f538ecac2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:02:42.594752 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Network hostable DHCP agents. Network: b17682f1-2ab2-4f30-acf6-e88f538ecac2, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 12:02:42.703693 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Network b17682f1-2ab2-4f30-acf6-e88f538ecac2 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 12:02:42.795172 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, b17682f1-2ab2-4f30-acf6-e88f538ecac2) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:02:42.804874 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e5b37cf8-276f-4336-b8d4-79d532c5431f None None] Network b17682f1-2ab2-4f30-acf6-e88f538ecac2 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:02:42.818505 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:42.819108 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:42.931065 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 862 time: 1.0390415 May 31 12:02:42.996900 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-175c8eb6-9830-4f82-aa3e-f3f931a36174 None None] Network b17682f1-2ab2-4f30-acf6-e88f538ecac2 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:02:43.068612 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38874) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:43.128162 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] Deleting subnet cfed87a0-8c41-4494-ae4b-5db44cdf1f92 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:02:43.131278 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:43.184232 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:43.246682 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:43.675569 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:43.679489 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c87f9c26-6445-49b6-92dc-a327b5f3df94 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:43.714593 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] Pushing event deleted for resources: {'Subnet': ['ID=cfed87a0-8c41-4494-ae4b-5db44cdf1f92,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:43.716838 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:43.721316 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:43.727126 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-15b32b0b-262b-4f21-81e6-3f9d9c1938ee admin admin] 10.222.0.48 "DELETE /v2.0/subnets/cfed87a0-8c41-4494-ae4b-5db44cdf1f92 HTTP/1.1" status: 204 len: 168 time: 0.6639905 May 31 12:02:43.729676 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38880) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:43.932394 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.057298 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.076871 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['25d565bb-afcb-4ac9-b31c-e0771872ad08']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:44.091829 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '5e68b53f-62bb-410e-82e8-536abd023714') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:44.092122 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '5e68b53f-62bb-410e-82e8-536abd023714') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:44.103397 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.118953 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.125428 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Releasing vlan 372 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:02:44.125831 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:02:44.139281 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:02:44.301708 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.305338 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:44.325587 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.331516 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Pushing event deleted for resources: {'Network': ['ID=25d565bb-afcb-4ac9-b31c-e0771872ad08,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:44.334258 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:44.340123 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] 10.222.0.48 "DELETE /v2.0/networks/25d565bb-afcb-4ac9-b31c-e0771872ad08 HTTP/1.1" status: 204 len: 168 time: 0.6048915 May 31 12:02:44.342160 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38886) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:44.402259 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] Deleting subnet a9fe49e6-a14f-4456-a88e-e3d144a0fef9 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:02:44.405883 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.456695 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.515951 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.835464 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.840815 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6aff0ef0-560d-4988-b2c0-0a157f374955 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:44.879391 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] Pushing event deleted for resources: {'Subnet': ['ID=a9fe49e6-a14f-4456-a88e-e3d144a0fef9,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:44.881350 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:44.883837 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:44.888242 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-0489a690-d134-4c88-8abf-c6ed1e57e83d admin admin] 10.222.0.48 "DELETE /v2.0/subnets/a9fe49e6-a14f-4456-a88e-e3d144a0fef9 HTTP/1.1" status: 204 len: 168 time: 0.5451601 May 31 12:02:44.890771 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38890) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:44.996934 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:45.119117 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:45.132030 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['b17682f1-2ab2-4f30-acf6-e88f538ecac2']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:45.144733 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'f009e1a2-b422-43cd-a0e1-da02b6b83476') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:45.145019 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'f009e1a2-b422-43cd-a0e1-da02b6b83476') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:02:45.156852 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:45.172667 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:45.178516 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Releasing vlan 357 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:02:45.178889 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:02:45.191782 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:02:45.358663 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:45.362099 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:45.376024 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:45.384787 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Pushing event deleted for resources: {'Network': ['ID=b17682f1-2ab2-4f30-acf6-e88f538ecac2,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:45.388552 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:45.393429 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] 10.222.0.48 "DELETE /v2.0/networks/b17682f1-2ab2-4f30-acf6-e88f538ecac2 HTTP/1.1" status: 204 len: 168 time: 0.4983976 May 31 12:02:45.797395 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38904) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:45.846973 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f293e649-5622-4259-9308-89d390abfd6f admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=d26c294be9c24ee4a83c99a700cf051e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0483036 May 31 12:02:45.849365 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 38910) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:45.907556 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-194de79a-2d50-49a1-88c8-ef2b36aa9599 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:45.934048 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-194de79a-2d50-49a1-88c8-ef2b36aa9599 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:46.075415 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-194de79a-2d50-49a1-88c8-ef2b36aa9599 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:46.080296 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1d11cf0f-cfe6-44ac-ad74-f3acc544fc48 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:46.092354 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-194de79a-2d50-49a1-88c8-ef2b36aa9599 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=55306038-4d32-415e-8d92-a66a660cafaf,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:46.095241 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-194de79a-2d50-49a1-88c8-ef2b36aa9599 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:46.097222 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-194de79a-2d50-49a1-88c8-ef2b36aa9599 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:46.100693 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-194de79a-2d50-49a1-88c8-ef2b36aa9599 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/55306038-4d32-415e-8d92-a66a660cafaf HTTP/1.1" status: 204 len: 168 time: 0.2479291 May 31 12:02:46.866684 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38920) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:46.915003 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-77023d00-eb4b-46fd-9cc1-f45bda40d0c3 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=2880850360b744be86bee375846ab5cd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0470462 May 31 12:02:46.917286 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 38924) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:46.976511 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-66bb5085-861c-48d0-9219-93f856626fc2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:47.005477 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-66bb5085-861c-48d0-9219-93f856626fc2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:47.161226 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-66bb5085-861c-48d0-9219-93f856626fc2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:47.166102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-afb35011-fee2-4b80-b760-5b125f051e10 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:47.180264 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-66bb5085-861c-48d0-9219-93f856626fc2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6e0bab21-9393-432d-bdfd-ef82101235c0,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:47.182728 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-66bb5085-861c-48d0-9219-93f856626fc2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:47.189093 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-66bb5085-861c-48d0-9219-93f856626fc2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:47.190216 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-66bb5085-861c-48d0-9219-93f856626fc2 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/6e0bab21-9393-432d-bdfd-ef82101235c0 HTTP/1.1" status: 204 len: 168 time: 0.2720146 May 31 12:02:49.219612 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-564e905a-6e0e-4710-adab-c74e8d2f203d req-3091baff-a6ef-4a24-8cd3-9fa273ae9985 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:49.221080 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-564e905a-6e0e-4710-adab-c74e8d2f203d req-3091baff-a6ef-4a24-8cd3-9fa273ae9985 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0838571 May 31 12:02:49.266135 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-564e905a-6e0e-4710-adab-c74e8d2f203d req-a86df40c-20be-420b-954d-16c97f6141ab tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0394864 May 31 12:02:50.470259 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-65eb32cc-c890-4b22-bbda-d890f4dcc671 req-533b6209-7e4c-4cbc-b7ec-96f2bf9be8c3 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:50.471699 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-65eb32cc-c890-4b22-bbda-d890f4dcc671 req-533b6209-7e4c-4cbc-b7ec-96f2bf9be8c3 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0793819 May 31 12:02:50.517123 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-65eb32cc-c890-4b22-bbda-d890f4dcc671 req-7b194c58-8ca1-4e05-a751-9c82472d8208 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0395598 May 31 12:02:50.941634 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8946e7bf-a68c-40e6-920f-bd14b1ceec50 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:51.706060 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-299e4523-c2e4-4349-ac59-3c4f9e3e045d req-2da3d2fb-892d-4657-8013-c76791b0df02 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:51.707504 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-299e4523-c2e4-4349-ac59-3c4f9e3e045d req-2da3d2fb-892d-4657-8013-c76791b0df02 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0834789 May 31 12:02:51.756583 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-299e4523-c2e4-4349-ac59-3c4f9e3e045d req-1d16c938-669b-439b-a209-a4f57cbd43c1 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0433209 May 31 12:02:53.046789 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53cd8cca-2731-4732-8346-78aa5ffba40e req-964d6f58-67cf-424b-a259-474de09c2777 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:53.048134 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-53cd8cca-2731-4732-8346-78aa5ffba40e req-964d6f58-67cf-424b-a259-474de09c2777 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.1842077 May 31 12:02:53.097146 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-53cd8cca-2731-4732-8346-78aa5ffba40e req-44fa9f80-e14b-4ddf-b0b6-0b0901f927e7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432725 May 31 12:02:53.327051 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 39020) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:53.391977 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.443554 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.457469 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:53.461315 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Usage tracker for resource:security_group_rule and tenant:d43e099565e4435da9439544406e01fc is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:53.464163 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:53.470956 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:53.524315 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-a7cb1a5c-55b8-442e-9fe2-970b21e53996 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.556380 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.649785 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.658355 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ac07b1fa-8748-4232-9f5c-c4eff12744f5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:53.696906 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=d43e099565e4435da9439544406e01fc&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3687744 May 31 12:02:53.699281 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6881c693-c961-4919-ab64-d164659bf709,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:53.700426 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39026) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:53.702554 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-34b3a073-441b-4c6f-8117-a09fb2a88e39 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:53.758326 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-79626345-3a2c-4280-bd0e-60c435eb7e26 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.785459 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-79626345-3a2c-4280-bd0e-60c435eb7e26 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.924127 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-79626345-3a2c-4280-bd0e-60c435eb7e26 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.929054 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-096cc375-132f-4801-ac17-0e2a9b6a91ac admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:53.943801 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-79626345-3a2c-4280-bd0e-60c435eb7e26 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6881c693-c961-4919-ab64-d164659bf709,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:53.946767 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-79626345-3a2c-4280-bd0e-60c435eb7e26 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:53.949415 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-79626345-3a2c-4280-bd0e-60c435eb7e26 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:53.950506 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-79626345-3a2c-4280-bd0e-60c435eb7e26 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/6881c693-c961-4919-ab64-d164659bf709 HTTP/1.1" status: 204 len: 168 time: 0.2490339 May 31 12:02:54.284381 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56e2a160-d508-4d8f-9ab9-4fb924ee704a req-f4e42c8a-2940-4cde-9448-0616a7ee3195 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:54.285860 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-56e2a160-d508-4d8f-9ab9-4fb924ee704a req-f4e42c8a-2940-4cde-9448-0616a7ee3195 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0836971 May 31 12:02:54.338217 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-56e2a160-d508-4d8f-9ab9-4fb924ee704a req-e890d10a-28dd-4bfd-aab8-4873e38592dc tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0467875 May 31 12:02:54.658137 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:54.734762 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39042) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:54.751023 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:54.765678 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:02:54.769473 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Usage tracker for resource:security_group_rule and tenant:228222fc1623413485e9788b94af69fa is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:02:54.772361 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:02:54.779020 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:02:54.859463 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:54.946807 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:54.955003 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-79626345-3a2c-4280-bd0e-60c435eb7e26 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:54.993859 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=84599617-d033-4389-a5ba-a052e0acb88b,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:55.000504 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=228222fc1623413485e9788b94af69fa&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2636950 May 31 12:02:55.000504 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:55.000799 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39048) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:02:55.060228 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0a80d351-bd9a-40c0-9316-44b649976b8a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:55.085851 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0a80d351-bd9a-40c0-9316-44b649976b8a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:55.238418 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0a80d351-bd9a-40c0-9316-44b649976b8a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:55.242199 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-5972d6e2-49ee-4b51-923d-ac277e2ee737 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:02:55.255909 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0a80d351-bd9a-40c0-9316-44b649976b8a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=84599617-d033-4389-a5ba-a052e0acb88b,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:02:55.258680 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0a80d351-bd9a-40c0-9316-44b649976b8a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:02:55.260529 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-0a80d351-bd9a-40c0-9316-44b649976b8a admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:55.262716 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-0a80d351-bd9a-40c0-9316-44b649976b8a admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/84599617-d033-4389-a5ba-a052e0acb88b HTTP/1.1" status: 204 len: 168 time: 0.2607858 May 31 12:02:55.518369 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5bd808f7-07f3-43c0-8b20-4df91239faf5 req-cca1161b-d561-41e5-ae84-24a82640513c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:55.519790 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5bd808f7-07f3-43c0-8b20-4df91239faf5 req-cca1161b-d561-41e5-ae84-24a82640513c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0810699 May 31 12:02:55.568628 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5bd808f7-07f3-43c0-8b20-4df91239faf5 req-3c649308-75dd-4dfc-95d9-1f7a2794ed21 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432472 May 31 12:02:56.092342 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:02:56.757097 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75aa6b94-8198-423b-b7de-5c3d02f0a10d req-c7f2136e-b258-4b0c-9631-036eb2f7daea tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:56.758564 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-75aa6b94-8198-423b-b7de-5c3d02f0a10d req-c7f2136e-b258-4b0c-9631-036eb2f7daea tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0808978 May 31 12:02:56.804652 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-75aa6b94-8198-423b-b7de-5c3d02f0a10d req-d5b249bf-6a4a-4b25-b881-72c8680a953d tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0395589 May 31 12:02:57.998629 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-13926450-bbde-4024-861f-a41a178c483f req-42f8c505-d17e-48f4-9986-776873feed30 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:58.000015 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-13926450-bbde-4024-861f-a41a178c483f req-42f8c505-d17e-48f4-9986-776873feed30 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0841181 May 31 12:02:58.045065 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-13926450-bbde-4024-861f-a41a178c483f req-49f9545c-4868-4fdd-8f4b-ce5af585cf39 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0393724 May 31 12:02:59.237530 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-037d8c90-ac5c-46d8-ae2d-b037a57f50cd req-c697d8d5-805f-4996-8c5d-72991e65521b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:59.239017 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-037d8c90-ac5c-46d8-ae2d-b037a57f50cd req-c697d8d5-805f-4996-8c5d-72991e65521b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0896230 May 31 12:02:59.288103 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-037d8c90-ac5c-46d8-ae2d-b037a57f50cd req-f7211d78-9afe-43b8-ab12-b06e6f513d56 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432215 May 31 12:02:59.367963 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-4dad2b64-fa46-435b-bb51-c47fc002ea2b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:59.369411 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-4dad2b64-fa46-435b-bb51-c47fc002ea2b service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1098 time: 0.0708015 May 31 12:02:59.505444 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-00a1979d-840d-484b-a3a5-e5d0b2f5b8fe service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:59.506932 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-00a1979d-840d-484b-a3a5-e5d0b2f5b8fe service neutron] 10.222.0.47 "GET /v2.0/networks?id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 200 len: 910 time: 0.1313393 May 31 12:02:59.613178 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-7d89571d-f9d9-42ac-987e-d36f2817b43c service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 200 len: 1001 time: 0.0977006 May 31 12:02:59.679238 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-edcb70eb-30d1-43c9-bb08-ac7c279d247d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:59.680636 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-edcb70eb-30d1-43c9-bb08-ac7c279d247d service neutron] 10.222.0.47 "GET /v2.0/subnets?id=3a19e015-3084-4732-bb65-ea124df3118f HTTP/1.1" status: 200 len: 848 time: 0.0607135 May 31 12:02:59.759048 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-03e5f4e5-8bc9-46dc-811c-41aeed1e8ddd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:02:59.761313 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-03e5f4e5-8bc9-46dc-811c-41aeed1e8ddd service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0736718 May 31 12:02:59.977787 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-23137242-8c4c-4acd-b9d7-3e055abf87c7 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2088490 May 31 12:03:00.111762 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fbdd4dba-6661-42ca-922a-837f071720a7 req-0eb4bdef-ac92-480c-9c9d-9e0a141f2e73 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1272392 May 31 12:03:00.469998 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bc8672d-8c4c-4794-bd24-940af0bb9d4d req-4223b825-a58e-4633-aba8-e84a67599172 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:00.471422 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8bc8672d-8c4c-4794-bd24-940af0bb9d4d req-4223b825-a58e-4633-aba8-e84a67599172 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0806243 May 31 12:03:00.515688 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8bc8672d-8c4c-4794-bd24-940af0bb9d4d req-9f65f876-1a1d-4eee-911d-0f27b367c9b7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0386579 May 31 12:03:01.695378 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-582fa9f8-7d94-416f-918b-be0fb4d2fefd req-560d0113-7ae6-4821-b591-447b86773738 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:01.696795 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-582fa9f8-7d94-416f-918b-be0fb4d2fefd req-560d0113-7ae6-4821-b591-447b86773738 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0776069 May 31 12:03:01.743540 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-582fa9f8-7d94-416f-918b-be0fb4d2fefd req-1dd2c50c-991d-4b1b-b26f-8df3756af508 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0406322 May 31 12:03:02.931310 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cee23070-2499-4bdc-86a0-0b86157fdd98 req-988d1b29-6fcc-4739-8a01-50714312a57c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:02.932948 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-cee23070-2499-4bdc-86a0-0b86157fdd98 req-988d1b29-6fcc-4739-8a01-50714312a57c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0827301 May 31 12:03:02.976196 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-cee23070-2499-4bdc-86a0-0b86157fdd98 req-7e8e7c0b-a486-4103-94f0-3fbdc388de13 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0378621 May 31 12:03:04.164166 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f325cbdb-4a33-4c6a-96ba-d374c643e036 req-7066e154-04f3-49a7-9317-2b891cefb830 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:04.165792 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f325cbdb-4a33-4c6a-96ba-d374c643e036 req-7066e154-04f3-49a7-9317-2b891cefb830 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0795984 May 31 12:03:04.210000 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f325cbdb-4a33-4c6a-96ba-d374c643e036 req-dcccb476-fa3d-460d-9ccf-a34c0290619f tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0387044 May 31 12:03:05.395981 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9db3e3b-3573-4946-9de1-ec2acf214fcf req-59164b46-33d1-49e4-9e82-dea892cdf2d9 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:05.397434 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a9db3e3b-3573-4946-9de1-ec2acf214fcf req-59164b46-33d1-49e4-9e82-dea892cdf2d9 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0803752 May 31 12:03:05.446194 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a9db3e3b-3573-4946-9de1-ec2acf214fcf req-cd65e671-c6f6-4b9e-9a30-e7a1a24f6682 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0430839 May 31 12:03:06.043805 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b4b147b-0791-4a69-87d1-fdbaae1188e6 req-ded216e6-aadc-4b1e-8992-d2f094ad9b83 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:06.045218 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0b4b147b-0791-4a69-87d1-fdbaae1188e6 req-ded216e6-aadc-4b1e-8992-d2f094ad9b83 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.1128905 May 31 12:03:06.108032 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0b4b147b-0791-4a69-87d1-fdbaae1188e6 req-d9c7c6fd-6638-4339-a61a-f41a161284cf tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0545228 May 31 12:03:06.117988 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-b78f4d2e-1fac-4e6e-98c2-e2c3494addc9 service neutron] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.1169438 May 31 12:03:07.307008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d44f462f-91f5-438f-9ea3-046835e12bbf req-0aa8c69b-0155-4006-988c-0a1d851ecbdc tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:07.308491 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d44f462f-91f5-438f-9ea3-046835e12bbf req-0aa8c69b-0155-4006-988c-0a1d851ecbdc tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0908120 May 31 12:03:07.352510 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d44f462f-91f5-438f-9ea3-046835e12bbf req-525880ce-8a3d-4407-b87c-7a80c520650d tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0381181 May 31 12:03:07.828955 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5add15c1-9f55-440c-b343-5d2f806a63c2 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:03:07.951870 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:07.977917 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:07.980220 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-dd464d85-b0b1-46ee-b616-5269eacd5d26" -d '{"events": [{"server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "name": "network-vif-unplugged", "status": "completed", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:07.988326 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:08.037047 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [207] Connection: close Content-Length: 182 Content-Type: application/json Date: Mon, 31 May 2021 12:03:07 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-de8a7ab1-ded1-414b-9484-371009b97085 x-openstack-request-id: req-de8a7ab1-ded1-414b-9484-371009b97085 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:08.037047 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "name": "network-vif-unplugged", "status": "failed", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2", "code": 422}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:08.037047 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-de8a7ab1-ded1-414b-9484-371009b97085 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:08.037047 dv-cinder-791239-4-iscsi neutron-server[99041]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'code': 422} returned with failed status May 31 12:03:08.055364 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:08.058920 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:08.125447 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=6']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:08.128719 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:08.544404 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-27b1a04f-0a2c-45d2-a359-b1aa93ca2401 req-d2bf8088-46b3-4ea0-8041-cec907e7bde0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:08.545781 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-27b1a04f-0a2c-45d2-a359-b1aa93ca2401 req-d2bf8088-46b3-4ea0-8041-cec907e7bde0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0801063 May 31 12:03:08.590336 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-27b1a04f-0a2c-45d2-a359-b1aa93ca2401 req-5dc225dc-91e2-484d-b98c-4c0fd784c401 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0386865 May 31 12:03:09.403839 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:09.735114 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 39232) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:03:09.790715 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7523f77-25a1-4078-91d9-32bca946bc44 req-43b57b7d-51e5-4358-a2f7-f732c49c6464 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:09.792298 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e7523f77-25a1-4078-91d9-32bca946bc44 req-43b57b7d-51e5-4358-a2f7-f732c49c6464 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0842819 May 31 12:03:09.840348 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e7523f77-25a1-4078-91d9-32bca946bc44 req-c7ce0d25-348d-4952-a197-9a1c718a8802 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0423038 May 31 12:03:09.860107 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:09.882700 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:09.887641 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Usage tracker for resource:security_group_rule and tenant:958a033b5b064c11aa4796b300bbb46c is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:03:09.890609 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:03:09.896272 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:09.980693 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:10.091958 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:10.099199 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f80e1a92-0d56-41dd-9ce6-777f98e50da7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:10.140730 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=958a033b5b064c11aa4796b300bbb46c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4045210 May 31 12:03:10.141769 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=78bf4411-382d-4afa-96ed-133c0fedbd85,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:10.143393 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39244) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:03:10.144950 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-986f1325-5824-4b07-8651-0b6414c60883 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:10.206829 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:10.232026 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:10.381000 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:10.384784 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-05a00d4b-cbaa-45c7-ae09-218aa0a348d8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:10.398268 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=78bf4411-382d-4afa-96ed-133c0fedbd85,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:10.406576 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:10.407673 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/78bf4411-382d-4afa-96ed-133c0fedbd85 HTTP/1.1" status: 204 len: 168 time: 0.2633326 May 31 12:03:10.409180 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.409547 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.409991 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.410387 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.410746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.411151 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.411500 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.411854 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.412187 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.412530 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.412869 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.413207 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:10.416382 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:10.447697 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-9d01af57-2ef2-4c53-a855-4c256810c214 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:10.449382 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-9d01af57-2ef2-4c53-a855-4c256810c214 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314&tenant_id=bbcb901221a14602a1f2f8b198287636 HTTP/1.1" status: 200 len: 1096 time: 0.0779903 May 31 12:03:10.548909 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-9b55f420-9906-410e-b992-9fb3a42db73c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:10.550412 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-9b55f420-9906-410e-b992-9fb3a42db73c service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1096 time: 0.0792630 May 31 12:03:10.688346 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-d0a4f2c1-1ad1-42de-a74b-a26b5783b88f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:10.689846 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-d0a4f2c1-1ad1-42de-a74b-a26b5783b88f service neutron] 10.222.0.47 "GET /v2.0/networks?id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 200 len: 910 time: 0.1329839 May 31 12:03:10.793060 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-3bd04bc3-2321-40e2-a157-d90843f39611 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 200 len: 999 time: 0.0964205 May 31 12:03:10.856181 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-5757c492-ff1c-4741-9339-3e11718a39bb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:10.857591 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-5757c492-ff1c-4741-9339-3e11718a39bb service neutron] 10.222.0.47 "GET /v2.0/subnets?id=3a19e015-3084-4732-bb65-ea124df3118f HTTP/1.1" status: 200 len: 848 time: 0.0573571 May 31 12:03:10.934279 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-f63c274b-3806-434e-9c1c-c0c79154be94 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:10.935760 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-f63c274b-3806-434e-9c1c-c0c79154be94 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0722907 May 31 12:03:11.039721 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39254) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:03:11.058559 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:11.072965 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:11.080112 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Usage tracker for resource:security_group_rule and tenant:5df4ead64d6445b0b13134f527b1cb41 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:03:11.083504 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:03:11.089837 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:11.136747 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-ebbd8afd-3205-427f-b811-396587a46521 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1940386 May 31 12:03:11.148136 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e1c3f8c-dd42-4c98-95a6-47d03c25c920 req-af95cc94-f4b9-4668-86c9-31397572b59a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:11.149586 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8e1c3f8c-dd42-4c98-95a6-47d03c25c920 req-af95cc94-f4b9-4668-86c9-31397572b59a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.1846287 May 31 12:03:11.179511 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:11.255044 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:11.264039 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0a80d351-bd9a-40c0-9316-44b649976b8a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:11.272913 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8e1c3f8c-dd42-4c98-95a6-47d03c25c920 req-baf7aa45-5aa2-4d90-9b0f-6675e2191e59 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1042769 May 31 12:03:11.317620 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=5df4ead64d6445b0b13134f527b1cb41&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2754173 May 31 12:03:11.317620 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=51d72ddc-f0c0-4ecd-8148-b6babf395698,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:11.319613 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39264) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:03:11.322351 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:11.334837 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e0d39c38-20b8-44be-8deb-8e71e30ab7af req-49969d8e-ebe3-4e0d-b9ac-1ad438ded743 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1841922 May 31 12:03:11.380678 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bf53a078-0a51-43d4-8b89-dda5e66a0d9d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:11.408548 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bf53a078-0a51-43d4-8b89-dda5e66a0d9d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:11.535107 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bf53a078-0a51-43d4-8b89-dda5e66a0d9d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:11.538911 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1eed6b99-ca04-44b8-a4cb-167f23c7c172 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:11.552099 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf53a078-0a51-43d4-8b89-dda5e66a0d9d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=51d72ddc-f0c0-4ecd-8148-b6babf395698,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:11.554999 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bf53a078-0a51-43d4-8b89-dda5e66a0d9d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:11.556928 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-bf53a078-0a51-43d4-8b89-dda5e66a0d9d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:11.558060 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-bf53a078-0a51-43d4-8b89-dda5e66a0d9d admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/51d72ddc-f0c0-4ecd-8148-b6babf395698 HTTP/1.1" status: 204 len: 168 time: 0.2374856 May 31 12:03:12.352502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39276) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:03:12.371808 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:12.436114 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:12.445105 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Usage tracker for resource:security_group_rule and tenant:6cf41e6c76b6456eb75a4958f9a2dadf is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:03:12.449618 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:03:12.460857 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:12.515719 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0db3580a-e807-472b-a731-d7e9b8461ea8 req-130711c4-995c-496e-a41b-d051e2c398dd tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:12.517173 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0db3580a-e807-472b-a731-d7e9b8461ea8 req-130711c4-995c-496e-a41b-d051e2c398dd tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.1242874 May 31 12:03:12.583553 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0db3580a-e807-472b-a731-d7e9b8461ea8 req-74222b39-9b68-449e-8c33-c9898b9c227a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0593276 May 31 12:03:12.610099 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:12.684260 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:12.692720 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-64b664c8-a517-4b2c-8b14-97e99eafbc8f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:12.731108 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=6cf41e6c76b6456eb75a4958f9a2dadf&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3773615 May 31 12:03:12.732737 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f3e2803d-76ea-4689-af2c-8d3e15fc0481,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:12.733799 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 39286) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:03:12.735660 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-02830012-06a9-4310-8dcd-b04fdf643e77 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:12.797479 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-65cb969c-0987-41ba-996e-b3f346175fc4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:12.916323 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-65cb969c-0987-41ba-996e-b3f346175fc4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:13.035919 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-65cb969c-0987-41ba-996e-b3f346175fc4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:13.040923 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bcd3c20c-832d-4eda-bfb4-37cc38c6736f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:13.055414 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-65cb969c-0987-41ba-996e-b3f346175fc4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f3e2803d-76ea-4689-af2c-8d3e15fc0481,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:13.057676 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-65cb969c-0987-41ba-996e-b3f346175fc4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:13.068040 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-65cb969c-0987-41ba-996e-b3f346175fc4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:13.069189 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-65cb969c-0987-41ba-996e-b3f346175fc4 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/f3e2803d-76ea-4689-af2c-8d3e15fc0481 HTTP/1.1" status: 204 len: 168 time: 0.3343451 May 31 12:03:13.774589 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73b45e43-c5c4-476d-9a03-24d632b43dde req-b2a33dc4-bb15-47b9-9576-f53c7ff0818a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:13.776905 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-73b45e43-c5c4-476d-9a03-24d632b43dde req-b2a33dc4-bb15-47b9-9576-f53c7ff0818a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0854516 May 31 12:03:13.821739 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-73b45e43-c5c4-476d-9a03-24d632b43dde req-3b358155-071a-4537-ab67-9bb91095c10b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391839 May 31 12:03:14.195231 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39318) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:03:14.298167 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-298787931-network', 'tenant_id': 'e946adaaf3ee413da7e5acaa60376c5b'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:03:14.302924 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:14.307635 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Usage tracker for resource:network and tenant:e946adaaf3ee413da7e5acaa60376c5b is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:03:14.310494 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:03:14.316719 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:14.428996 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Made reservation on behalf of e946adaaf3ee413da7e5acaa60376c5b for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:03:14.433839 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:14.445279 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:14.459844 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:14.463642 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Usage tracker for resource:security_group_rule and tenant:e946adaaf3ee413da7e5acaa60376c5b is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:03:14.466452 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:03:14.472161 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:14.553359 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:14.656802 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:14.665055 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1231f953-8cd6-476b-af09-c6caa0ece573 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:14.702843 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] vlan segment allocate from pool success with {'vlan_id': 354, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 12:03:14.712794 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2e80e113-7a5f-431b-b1bc-fdc03edd754f,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:14.717490 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:14.725191 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:14.725516 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Added segment 696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90 of type vlan for network 1b0e8409-da86-4bc0-b712-3072314a6793 May 31 12:03:14.759184 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:14.769780 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74469cfc-aecc-425e-8805-5f43b181eb9e req-6821b2cd-3371-43f3-9cc4-b94e6e3482c7 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:14.771219 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74469cfc-aecc-425e-8805-5f43b181eb9e req-6821b2cd-3371-43f3-9cc4-b94e6e3482c7 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 946 time: 0.1535263 May 31 12:03:14.829862 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-74469cfc-aecc-425e-8805-5f43b181eb9e req-3ebf45bb-1d38-4fb0-9d36-5befc2612f8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0522351 May 31 12:03:14.886694 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:14.888473 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:14.889061 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:14.892578 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:14.998908 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Pushing event updated for resources: {'Network': ['ID=1b0e8409-da86-4bc0-b712-3072314a6793,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:15.001857 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:15.008451 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 1b0e8409-da86-4bc0-b712-3072314a6793 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 12:03:15.017844 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9644a4a7-da4f-422a-8f49-e8971cf25f84 req-4cb3efad-e868-4b25-ae64-d90e1f371b24 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:15.019283 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9644a4a7-da4f-422a-8f49-e8971cf25f84 req-4cb3efad-e868-4b25-ae64-d90e1f371b24 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0883765 May 31 12:03:15.068150 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9644a4a7-da4f-422a-8f49-e8971cf25f84 req-0137f121-0b7a-42eb-99db-1a341d1d37f4 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0429356 May 31 12:03:15.073367 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a5b4a737-2f3d-4c28-8c5f-619826094d9f admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 0.8769186 May 31 12:03:15.076104 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 39328) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:03:15.084414 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Request body: {'subnet': {'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-298787931-subnet', 'tenant_id': 'e946adaaf3ee413da7e5acaa60376c5b', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:03:15.087673 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:15.092303 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Usage tracker for resource:subnet and tenant:e946adaaf3ee413da7e5acaa60376c5b is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:03:15.095268 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:03:15.101253 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:15.183106 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Made reservation on behalf of e946adaaf3ee413da7e5acaa60376c5b for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:03:15.185418 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:15.734683 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:15.738851 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e60fe78d-274c-4a85-b5b7-3551b286bfb1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:15.809678 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Pushing event updated for resources: {'Subnet': ['ID=1d9b77a2-8a70-4491-974c-8c516e56cff2,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:15.812240 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:15.819123 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Checking segment: {'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:03:15.820290 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Network hostable DHCP agents. Network: 1b0e8409-da86-4bc0-b712-3072314a6793, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 12:03:15.974142 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Network 1b0e8409-da86-4bc0-b712-3072314a6793 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 12:03:16.023010 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a228e04-4b2f-43bf-8152-b767bf43ffcc req-ae649c64-e2fa-42e9-af70-57e32dccdab5 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:16.024448 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9a228e04-4b2f-43bf-8152-b767bf43ffcc req-ae649c64-e2fa-42e9-af70-57e32dccdab5 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 946 time: 0.0897448 May 31 12:03:16.072104 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9a228e04-4b2f-43bf-8152-b767bf43ffcc req-6a8d3975-6002-4483-a380-fcde3077e08a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409944 May 31 12:03:16.073636 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 1b0e8409-da86-4bc0-b712-3072314a6793) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:03:16.083023 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:16.083795 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:16.085346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-bf438307-7518-4b66-87fe-502bac74464c None None] Network 1b0e8409-da86-4bc0-b712-3072314a6793 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:03:16.085841 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-6d99717e-0785-4c18-a058-fed0e6d29ed0 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:16.096748 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-6d99717e-0785-4c18-a058-fed0e6d29ed0 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.47 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 946 time: 0.0916455 May 31 12:03:16.220625 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a0f791e2-0740-410b-ac53-bde32a636592 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.1433773 May 31 12:03:16.297721 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6024d656-6740-4fda-b7ee-4db644680c58 None None] Network 1b0e8409-da86-4bc0-b712-3072314a6793 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:03:16.302376 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Deleting port a2d3edee-28b8-4d43-9cd4-e905470dbb33 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:03:16.302716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:16.445275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-36783c01-d75e-4760-af84-211eb0cc43a9 req-234b5338-a8aa-4f2b-949c-85fc461187e9 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:16.446745 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-36783c01-d75e-4760-af84-211eb0cc43a9 req-234b5338-a8aa-4f2b-949c-85fc461187e9 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.2494452 May 31 12:03:16.448995 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] For port a2d3edee-28b8-4d43-9cd4-e905470dbb33, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a2d3edee-28b8-4d43-9cd4-e905470dbb33,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:16.449347 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:16.581205 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Calling delete_port for a2d3edee-28b8-4d43-9cd4-e905470dbb33 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:03:16.630322 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-36783c01-d75e-4760-af84-211eb0cc43a9 req-1c42804f-a659-4877-b8db-5ef26c2dcd7b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0563722 May 31 12:03:16.800605 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:16.808274 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:16.815268 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['a2d3edee-28b8-4d43-9cd4-e905470dbb33'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:03:16.857902 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Pushing event deleted for resources: {'Port': ['ID=a2d3edee-28b8-4d43-9cd4-e905470dbb33,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:16.861498 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:16.932685 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] No DVR routers for this DVR port a2d3edee-28b8-4d43-9cd4-e905470dbb33 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:03:16.935197 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:16.937759 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9c533e31-9cae-413c-a2e1-1be300a4878e', 'name': 'network-vif-deleted', 'tag': 'a2d3edee-28b8-4d43-9cd4-e905470dbb33'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:16.940022 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-dbd22d32-a97b-4cee-8714-09ec3e916856" -d '{"events": [{"server_uuid": "9c533e31-9cae-413c-a2e1-1be300a4878e", "name": "network-vif-deleted", "tag": "a2d3edee-28b8-4d43-9cd4-e905470dbb33"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:16.984342 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:03:16 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-b2359260-6dbf-462b-a1e3-18271a1c2e58 x-openstack-request-id: req-b2359260-6dbf-462b-a1e3-18271a1c2e58 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:16.984663 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "9c533e31-9cae-413c-a2e1-1be300a4878e", "name": "network-vif-deleted", "tag": "a2d3edee-28b8-4d43-9cd4-e905470dbb33", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:16.984945 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-b2359260-6dbf-462b-a1e3-18271a1c2e58 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:16.985369 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9c533e31-9cae-413c-a2e1-1be300a4878e', 'name': 'network-vif-deleted', 'tag': 'a2d3edee-28b8-4d43-9cd4-e905470dbb33', 'status': 'completed', 'code': 200} May 31 12:03:17.006712 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9bedae51-cd19-4f84-95af-8a49f861ce09 req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.47 "DELETE /v2.0/ports/a2d3edee-28b8-4d43-9cd4-e905470dbb33 HTTP/1.1" status: 204 len: 173 time: 0.9048705 May 31 12:03:17.227509 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-d3aa4f29-b667-4d8a-adbd-3e56cf54dbf6 req-1c2ad7a9-7eae-41e1-895e-669ad9274c35 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Synchronizing usage tracker for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:03:17.300652 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-d3aa4f29-b667-4d8a-adbd-3e56cf54dbf6 req-1c2ad7a9-7eae-41e1-895e-669ad9274c35 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Unset dirty status for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:03:17.301850 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d3aa4f29-b667-4d8a-adbd-3e56cf54dbf6 req-1c2ad7a9-7eae-41e1-895e-669ad9274c35 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=9c533e31-9cae-413c-a2e1-1be300a4878e HTTP/1.1" status: 200 len: 210 time: 0.1198735 May 31 12:03:17.464803 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:03:17.818037 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2d9bcc0c-27bf-467a-8c8b-4ca867d67937 req-41f9d43d-adaf-4c04-9434-543faad05154 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:17.819611 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2d9bcc0c-27bf-467a-8c8b-4ca867d67937 req-41f9d43d-adaf-4c04-9434-543faad05154 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 943 time: 0.0835900 May 31 12:03:17.855229 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5add15c1-9f55-440c-b343-5d2f806a63c2 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:03:17.868503 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2d9bcc0c-27bf-467a-8c8b-4ca867d67937 req-bf03c2c2-a25a-496e-8418-4ee556b4e5c8 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0431218 May 31 12:03:17.999983 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:18.014755 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'port_id': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'mac_address': 'fa:16:3e:dc:71:13', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 397, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:03:18.069030 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:18.085074 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 5add15c1-9f55-440c-b343-5d2f806a63c2 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:03:18.100571 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:18.160555 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:18.162351 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:18.229393 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=7']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:18.232989 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:18.351806 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['5add15c1-9f55-440c-b343-5d2f806a63c2'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:03:18.678505 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-84d4d879-9910-4cc5-88ab-f9e00ea596dc tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:18.679174 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-84d4d879-9910-4cc5-88ab-f9e00ea596dc tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Synchronizing usage tracker for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:network {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:03:18.786282 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-84d4d879-9910-4cc5-88ab-f9e00ea596dc tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Unset dirty status for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:network {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:03:18.787492 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-84d4d879-9910-4cc5-88ab-f9e00ea596dc tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/networks?id=1b0e8409-da86-4bc0-b712-3072314a6793 HTTP/1.1" status: 200 len: 808 time: 0.2656105 May 31 12:03:18.802305 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-3295e658-1510-42ad-8de9-ff0d5e1c3319 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/quotas/e946adaaf3ee413da7e5acaa60376c5b HTTP/1.1" status: 200 len: 375 time: 0.0091918 May 31 12:03:18.852431 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-4cedfd88-dcd4-4850-bb23-1dae56b63c28 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0446339 May 31 12:03:18.992826 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-b0db7f9d-b2f2-4552-9b65-af485208e623 service neutron] 10.222.0.48 "GET /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1335781 May 31 12:03:19.060905 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d41624e-94d3-49e5-886a-2922b7b4d168 req-a1bc70dd-23ee-4146-a685-6c9b0a2e7322 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:19.062393 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1d41624e-94d3-49e5-886a-2922b7b4d168 req-a1bc70dd-23ee-4146-a685-6c9b0a2e7322 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 944 time: 0.0816824 May 31 12:03:19.108994 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1d41624e-94d3-49e5-886a-2922b7b4d168 req-7738f8ee-6759-46e6-83f2-5d372d8377cd tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0407772 May 31 12:03:19.126197 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-a4a436e5-f171-44c7-98ab-68b0f10fcacb service neutron] 10.222.0.48 "GET /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1275356 May 31 12:03:19.404993 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-e60d8bc2-f607-4fdc-8b28-6ae4632d9e44 req-19d49f6d-51bf-4ddf-a5de-3991ab0aa84f tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 210 time: 0.0462956 May 31 12:03:20.167899 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5add15c1-9f55-440c-b343-5d2f806a63c2 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:03:20.239832 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 5add15c1-9f55-440c-b343-5d2f806a63c2 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:03:20.240254 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:20.299546 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6388bd79-d1db-4eca-8d6b-ab3e62ced733 req-33545e53-5498-4612-859c-53977f42546c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:20.301029 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6388bd79-d1db-4eca-8d6b-ab3e62ced733 req-33545e53-5498-4612-859c-53977f42546c tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 944 time: 0.0830493 May 31 12:03:20.350687 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6388bd79-d1db-4eca-8d6b-ab3e62ced733 req-059480a5-3542-4130-8a57-da25dcbc2fc0 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0437808 May 31 12:03:20.393287 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:20.421474 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:20.423698 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-2de5d752-47c4-4f98-ab16-af3de1d31638" -d '{"events": [{"server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "name": "network-vif-plugged", "status": "completed", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:20.431321 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:20.467957 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:03:20 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-75698584-e440-488a-9ecf-8bf1feabc0cd x-openstack-request-id: req-75698584-e440-488a-9ecf-8bf1feabc0cd {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:20.468264 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "name": "network-vif-plugged", "status": "completed", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:20.468559 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-75698584-e440-488a-9ecf-8bf1feabc0cd {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:20.468973 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'code': 200} May 31 12:03:20.506627 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-331616c0-088b-4850-9c69-39e34a060a5b req-8720def7-e7e8-40f8-a5cf-e62496df6244 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 210 time: 0.0432122 May 31 12:03:20.558468 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:20.559512 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device a2d3edee-28b8-4d43-9cd4-e905470dbb33 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:03:20.582982 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-59e45fae-cbb7-40be-b86d-aa361883d50f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:20.629966 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: a2d3edee-28b8-4d43-9cd4-e905470dbb33 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:03:20.630321 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device a2d3edee-28b8-4d43-9cd4-e905470dbb33 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:03:20.655832 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=8']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:20.658660 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:20.892220 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-83da3da5-0a06-4693-9304-1376cdd67f3c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:21.536550 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ee5e522-7c07-486c-b562-1c0fe60904dc req-b26fbfcf-73df-44a0-b516-d557996c66d6 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:21.538013 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2ee5e522-7c07-486c-b562-1c0fe60904dc req-b26fbfcf-73df-44a0-b516-d557996c66d6 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0806546 May 31 12:03:21.582759 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2ee5e522-7c07-486c-b562-1c0fe60904dc req-bd284ebf-691c-4602-9471-d0726f3fe1fa tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0385666 May 31 12:03:21.647643 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-96be35d8-af99-4ee1-b4f7-6a41e56d4502 req-d1670021-43d9-4de8-9b08-4dfa3c39ed36 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 210 time: 0.0421147 May 31 12:03:22.470644 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d5b3b4f-5cb1-418b-9b37-bc6f4018f568 req-4977453d-f85f-4cef-a724-86a45695de45 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:22.472141 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7d5b3b4f-5cb1-418b-9b37-bc6f4018f568 req-4977453d-f85f-4cef-a724-86a45695de45 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1098 time: 0.0759909 May 31 12:03:22.517643 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-7d5b3b4f-5cb1-418b-9b37-bc6f4018f568 req-9bf17a4f-8ec4-4fb6-8ef1-176680f0964d service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0366256 May 31 12:03:22.778090 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0bef73e3-b3fc-45ba-80f6-71580a8f8833 req-61585045-06f9-4810-9450-e2e145f67375 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:22.779555 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0bef73e3-b3fc-45ba-80f6-71580a8f8833 req-61585045-06f9-4810-9450-e2e145f67375 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0850544 May 31 12:03:22.797987 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ec0728f4-8e87-49f8-98a9-74b246246fbc req-47adefd5-22c9-4949-b757-ce8f655bf7a2 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 210 time: 0.0434651 May 31 12:03:22.825482 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0bef73e3-b3fc-45ba-80f6-71580a8f8833 req-95e79277-b164-4d73-bbce-085030d90181 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0402689 May 31 12:03:23.457955 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:23.492147 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-38df32cf-785a-4099-af44-25899121e2b4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:23.561026 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-882bfde5-470d-4ebf-8f1c-85ed4db0aa26 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:23.562396 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-882bfde5-470d-4ebf-8f1c-85ed4db0aa26 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.47 "GET /v2.0/networks?id=1b0e8409-da86-4bc0-b712-3072314a6793 HTTP/1.1" status: 200 len: 808 time: 0.1655059 May 31 12:03:23.575356 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Request body: {'port': {'device_id': '7fd6c48b-9aad-4c92-bbef-75cc9871eb5c', 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793', 'admin_state_up': True, 'tenant_id': 'e946adaaf3ee413da7e5acaa60376c5b'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:03:23.792443 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:23.796253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Usage tracker for resource:port and tenant:e946adaaf3ee413da7e5acaa60376c5b is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:03:23.799074 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:03:23.804662 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:23.883041 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Made reservation on behalf of e946adaaf3ee413da7e5acaa60376c5b for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:03:23.887827 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:23.996796 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-42539c65-35da-4c58-9447-da40ef3b8090 req-1fabca4e-6d09-4954-a360-10c4b1989d6d tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 210 time: 0.0879896 May 31 12:03:24.019343 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a13b5edd-446d-459c-bfb0-c1afe9d545c6 req-0a70bac3-9c06-44bb-92f9-6b19ebcdb9db tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:24.021559 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a13b5edd-446d-459c-bfb0-c1afe9d545c6 req-0a70bac3-9c06-44bb-92f9-6b19ebcdb9db tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0869641 May 31 12:03:24.067682 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a13b5edd-446d-459c-bfb0-c1afe9d545c6 req-71581afb-272d-4a9b-aa7c-653300f639c2 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0398462 May 31 12:03:24.109436 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Allocated IP 172.20.0.8 (1b0e8409-da86-4bc0-b712-3072314a6793/1d9b77a2-8a70-4491-974c-8c516e56cff2/e4837294-fbd4-4d90-a6c0-e9caa29a5b28) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:03:24.307273 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:24.400620 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:24.406039 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ab93f581-2bf3-4f2f-a626-45e877077f7d tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:24.470053 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Network 1b0e8409-da86-4bc0-b712-3072314a6793 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:03:24.473100 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:03:24.481394 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:03:24.513712 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Pushing event updated for resources: {'Port': ['ID=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:24.521010 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:24.548854 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:24.549515 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:24.643529 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:24.675949 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.1050923 May 31 12:03:24.688148 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Request body: {'port': {'device_id': '7fd6c48b-9aad-4c92-bbef-75cc9871eb5c', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:03:24.796644 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:25.210659 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e4837294-fbd4-4d90-a6c0-e9caa29a5b28 {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:03:25.220216 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:25.240278 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Transition to ACTIVE for port object e4837294-fbd4-4d90-a6c0-e9caa29a5b28 will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:03:25.256057 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95b08f72-1419-4c12-abca-75bba9382496 req-3a147d9e-25d4-440f-843f-2e5130902843 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:25.256883 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-95b08f72-1419-4c12-abca-75bba9382496 req-3a147d9e-25d4-440f-843f-2e5130902843 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Synchronizing usage tracker for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:03:25.303530 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:25.309166 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b62194a5-286d-4ffd-a351-4ad99e0b085e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:25.364813 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-95b08f72-1419-4c12-abca-75bba9382496 req-3a147d9e-25d4-440f-843f-2e5130902843 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Unset dirty status for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:03:25.365994 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-95b08f72-1419-4c12-abca-75bba9382496 req-3a147d9e-25d4-440f-843f-2e5130902843 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 931 time: 0.2291520 May 31 12:03:25.512422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Pushing event updated for resources: {'Port': ['ID=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:25.515168 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.206s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:25.551328 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-95b08f72-1419-4c12-abca-75bba9382496 req-c56772dc-b322-42bb-9d3f-899077b6565f tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1297798 May 31 12:03:25.590543 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Attempting to bind port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:03:25.590865 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Attempting to bind port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:03:25.591186 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Attempting to bind port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 on network 1b0e8409-da86-4bc0-b712-3072314a6793 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:03:25.595702 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 on network 1b0e8409-da86-4bc0-b712-3072314a6793 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 12:03:25.596008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Attempting to bind port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 on network 1b0e8409-da86-4bc0-b712-3072314a6793 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:03:25.600983 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 3, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:03:25.601308 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Checking segment: {'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:03:25.601707 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Bound using segment: {'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:03:25.602118 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Bound port: e4837294-fbd4-4d90-a6c0-e9caa29a5b28, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:03:25.602455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:25.697887 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] For port e4837294-fbd4-4d90-a6c0-e9caa29a5b28, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:03:25.700783 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] For port e4837294-fbd4-4d90-a6c0-e9caa29a5b28, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:03:25.792131 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:25.795663 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cec91fe8-8f63-478b-8cf9-a1695b29390b tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:25.852690 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:25.854364 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:25.855331 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] 10.222.0.47 "PUT /v2.0/ports/e4837294-fbd4-4d90-a6c0-e9caa29a5b28 HTTP/1.1" status: 200 len: 1093 time: 1.1712506 May 31 12:03:25.855823 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7fd6c48b-9aad-4c92-bbef-75cc9871eb5c', 'tag': 'e4837294-fbd4-4d90-a6c0-e9caa29a5b28'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:25.858073 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-4976b477-c6bc-441c-84bf-c221c85a434f" -d '{"events": [{"name": "network-changed", "server_uuid": "7fd6c48b-9aad-4c92-bbef-75cc9871eb5c", "tag": "e4837294-fbd4-4d90-a6c0-e9caa29a5b28"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:25.875578 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Pushing event updated for resources: {'Port': ['ID=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:25.878649 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:25.899999 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:03:25 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-a1ce6468-ada8-480f-881f-a624fec88fa6 x-openstack-request-id: req-a1ce6468-ada8-480f-881f-a624fec88fa6 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:25.900393 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "7fd6c48b-9aad-4c92-bbef-75cc9871eb5c", "tag": "e4837294-fbd4-4d90-a6c0-e9caa29a5b28", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:25.900670 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-a1ce6468-ada8-480f-881f-a624fec88fa6 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:25.901102 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7fd6c48b-9aad-4c92-bbef-75cc9871eb5c', 'tag': 'e4837294-fbd4-4d90-a6c0-e9caa29a5b28', 'status': 'completed', 'code': 200} May 31 12:03:26.117489 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:26.824439 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc7df49d-88e4-4978-823c-f46ea9a8ecf9 req-deeec2bc-1b9c-4ba0-960d-9078f72d3a4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:26.825901 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-bc7df49d-88e4-4978-823c-f46ea9a8ecf9 req-deeec2bc-1b9c-4ba0-960d-9078f72d3a4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 943 time: 0.1707680 May 31 12:03:26.870095 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-bc7df49d-88e4-4978-823c-f46ea9a8ecf9 req-e64e1746-a525-49da-b6fb-d19ec1dc50d7 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0379171 May 31 12:03:28.061204 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-24d7a49a-2974-4c92-8125-a5649ac548f3 req-e1bc94e0-3bb1-491c-8b41-c64043117ede tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:28.062669 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-24d7a49a-2974-4c92-8125-a5649ac548f3 req-e1bc94e0-3bb1-491c-8b41-c64043117ede tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 943 time: 0.0850692 May 31 12:03:28.109805 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-24d7a49a-2974-4c92-8125-a5649ac548f3 req-f9dfacfd-5082-4bd2-bb72-6bf25d9eab40 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Synchronizing usage tracker for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:security_group {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:03:28.194633 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-24d7a49a-2974-4c92-8125-a5649ac548f3 req-f9dfacfd-5082-4bd2-bb72-6bf25d9eab40 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Unset dirty status for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:security_group {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:03:28.195785 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-24d7a49a-2974-4c92-8125-a5649ac548f3 req-f9dfacfd-5082-4bd2-bb72-6bf25d9eab40 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1273170 May 31 12:03:28.486691 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4976b477-c6bc-441c-84bf-c221c85a434f req-e742937a-8426-4d54-b2cf-145daf578446 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:28.488042 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4976b477-c6bc-441c-84bf-c221c85a434f req-e742937a-8426-4d54-b2cf-145daf578446 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 1096 time: 0.0745864 May 31 12:03:28.631576 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4976b477-c6bc-441c-84bf-c221c85a434f req-2fbadaa8-d97a-449d-9bf7-1cf08141ef4e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:28.633049 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4976b477-c6bc-441c-84bf-c221c85a434f req-2fbadaa8-d97a-449d-9bf7-1cf08141ef4e service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&shared=False HTTP/1.1" status: 200 len: 909 time: 0.1347902 May 31 12:03:28.773984 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4976b477-c6bc-441c-84bf-c221c85a434f req-3c59a434-70e0-410d-8e9d-a6d790e359bf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:28.775456 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4976b477-c6bc-441c-84bf-c221c85a434f req-3c59a434-70e0-410d-8e9d-a6d790e359bf service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1360846 May 31 12:03:28.885245 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-b40d2854-52f8-48d6-9939-0f6bef716743 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:28.886981 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-b40d2854-52f8-48d6-9939-0f6bef716743 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 1096 time: 0.0746162 May 31 12:03:28.956983 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-e1b19e91-e8b6-4366-b421-f2763bf92568 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=e4837294-fbd4-4d90-a6c0-e9caa29a5b28 HTTP/1.1" status: 200 len: 217 time: 0.0635226 May 31 12:03:29.020075 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-8ac7245b-5a85-47cf-b391-d98321c181de service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:29.021027 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-8ac7245b-5a85-47cf-b391-d98321c181de service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1d9b77a2-8a70-4491-974c-8c516e56cff2 HTTP/1.1" status: 200 len: 848 time: 0.0578227 May 31 12:03:29.068102 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-a874995c-3e99-4059-b0e1-885e2e58a73b service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=1b0e8409-da86-4bc0-b712-3072314a6793&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0409873 May 31 12:03:29.200564 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-d711f156-ddc4-4efc-9c43-cffa3a18dbba service neutron] 10.222.0.47 "GET /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1264746 May 31 12:03:29.387128 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4991754e-e16d-4401-be35-f8b59eed7c24 req-17d8ca44-56ef-4499-96f2-f1d96ac567bc service neutron] 10.222.0.47 "GET /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1801569 May 31 12:03:29.419066 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b3a889b-b226-4e33-983e-846cfeae81ad req-072008ee-5cc8-489f-9567-3b60db9946b4 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:29.420495 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5b3a889b-b226-4e33-983e-846cfeae81ad req-072008ee-5cc8-489f-9567-3b60db9946b4 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 943 time: 0.1089454 May 31 12:03:29.465946 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5b3a889b-b226-4e33-983e-846cfeae81ad req-82f549a5-0fc4-4916-9761-7f20496e64eb tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397632 May 31 12:03:29.640616 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-1fcb7c11-7bb5-436a-b1ec-5a9941ee2aa9 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:29.642072 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-1fcb7c11-7bb5-436a-b1ec-5a9941ee2aa9 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/networks?id=ad7523f0-ba37-4688-b7be-179e0e8eae53 HTTP/1.1" status: 200 len: 814 time: 0.2394650 May 31 12:03:29.656867 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-9c78fff7-b8d9-41f9-bea3-8e9a923088f9 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/quotas/e0fbc2aa94f84d079cbe91bba91e2516 HTTP/1.1" status: 200 len: 375 time: 0.0093973 May 31 12:03:29.703018 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-05dd853d-42fd-4057-8a9b-b0279b9913dc tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0411782 May 31 12:03:29.832765 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-dd5e0462-b7c5-4286-bcbd-4988afb052ff service neutron] 10.222.0.48 "GET /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1232100 May 31 12:03:29.963696 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e71f9316-0140-466c-8fb8-cefab7fe1940 service neutron] 10.222.0.48 "GET /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1237364 May 31 12:03:30.271123 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-fb79d8bc-0d5b-4f91-8861-959f1c0c983f req-f7d95ec2-12bf-450a-b4d1-d0e00a527ae6 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 210 time: 0.0440316 May 31 12:03:30.664448 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9185128-626e-4a62-b23b-7e4dc68c9381 req-2b8521ff-4308-444c-b06a-634d31037248 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:30.666541 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a9185128-626e-4a62-b23b-7e4dc68c9381 req-2b8521ff-4308-444c-b06a-634d31037248 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 943 time: 0.0880384 May 31 12:03:30.711794 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a9185128-626e-4a62-b23b-7e4dc68c9381 req-6135ae58-e0ee-4b9c-b2a7-1f3394838def tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0392089 May 31 12:03:31.426068 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9ccebb24-bbfb-47f5-8a3e-c8c408e160d6 req-f6f8eaed-aa7a-4832-898f-d4ff85d1374f tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 210 time: 0.0457187 May 31 12:03:31.898450 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d63a99ce-b663-42e6-8ddd-2b0e22b60eb3 req-b92ee24b-57b6-4663-9ab8-7036a0bb7f9c tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:31.899903 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d63a99ce-b663-42e6-8ddd-2b0e22b60eb3 req-b92ee24b-57b6-4663-9ab8-7036a0bb7f9c tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 943 time: 0.0820222 May 31 12:03:31.944443 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d63a99ce-b663-42e6-8ddd-2b0e22b60eb3 req-0df89bef-5186-4e4f-9bf1-1f96a89542a1 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0385568 May 31 12:03:32.302790 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-9057f886-8c8c-4220-9941-0a17d0fff322 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:32.304145 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-9057f886-8c8c-4220-9941-0a17d0fff322 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.47 "GET /v2.0/networks?id=ad7523f0-ba37-4688-b7be-179e0e8eae53 HTTP/1.1" status: 200 len: 814 time: 0.1447141 May 31 12:03:32.315978 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Request body: {'port': {'device_id': '55072c0c-2294-4520-886d-0beec11f9b50', 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53', 'admin_state_up': True, 'tenant_id': 'e0fbc2aa94f84d079cbe91bba91e2516'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:03:32.455815 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:32.466317 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:32.510176 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e4837294-fbd4-4d90-a6c0-e9caa29a5b28 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:03:32.577934 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3bc2f8c8-fef6-47df-9410-2aee8d244e83 req-b70c6b40-bcc8-4549-b03e-243e7b21a00a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 210 time: 0.0431504 May 31 12:03:32.606239 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Made reservation on behalf of e0fbc2aa94f84d079cbe91bba91e2516 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:03:32.609833 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:32.646078 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e4837294-fbd4-4d90-a6c0-e9caa29a5b28, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:32.663730 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'e4837294-fbd4-4d90-a6c0-e9caa29a5b28', 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793', 'port_id': 'e4837294-fbd4-4d90-a6c0-e9caa29a5b28', 'mac_address': 'fa:16:3e:1b:7d:49', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 354, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1d9b77a2-8a70-4491-974c-8c516e56cff2', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:03:32.721237 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:32.737953 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id e4837294-fbd4-4d90-a6c0-e9caa29a5b28 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:03:32.753745 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e4837294-fbd4-4d90-a6c0-e9caa29a5b28, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:32.852346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:32.852346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:32.860683 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Allocated IP 172.20.0.5 (ad7523f0-ba37-4688-b7be-179e0e8eae53/e4a37872-4de9-422e-a54c-71186b6f99d7/5085dbf1-e506-4928-8a5c-fa72496efeb5) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:03:32.917896 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:32.925610 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.925952 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.926300 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.926623 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.926959 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.927285 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.927689 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.928014 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.928338 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.928660 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.928997 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.929298 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:32.934328 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:33.067523 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:33.073581 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['e4837294-fbd4-4d90-a6c0-e9caa29a5b28'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:03:33.132287 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03fc4cb7-4728-4c8c-ba63-02cd4effaf9b req-894db23e-1ebf-4b92-a0aa-0a0ad4694740 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:33.133689 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-03fc4cb7-4728-4c8c-ba63-02cd4effaf9b req-894db23e-1ebf-4b92-a0aa-0a0ad4694740 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 944 time: 0.0850568 May 31 12:03:33.157771 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:33.164468 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:33.184925 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-03fc4cb7-4728-4c8c-ba63-02cd4effaf9b req-69bd8322-ac86-49cc-96a7-8c71e696d372 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0450366 May 31 12:03:33.233192 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Network ad7523f0-ba37-4688-b7be-179e0e8eae53 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:03:33.236387 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:03:33.240350 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:03:33.279504 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Pushing event updated for resources: {'Port': ['ID=5085dbf1-e506-4928-8a5c-fa72496efeb5,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:33.296773 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.297128 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.297490 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.297873 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.298228 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.298585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.298935 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.299286 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.299631 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.299975 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.300319 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.300662 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:03:33.305617 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.141s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:33.318040 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:33.318638 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:33.426781 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.1142859 May 31 12:03:33.438836 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Request body: {'port': {'device_id': '55072c0c-2294-4520-886d-0beec11f9b50', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:03:33.550095 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:33.873412 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b76a81fe-e1be-40f5-973e-2481d13c2aff req-cb82ffc4-bbc0-48e5-96c6-78f70ee0b1d8 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:33.874031 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-b76a81fe-e1be-40f5-973e-2481d13c2aff req-cb82ffc4-bbc0-48e5-96c6-78f70ee0b1d8 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Synchronizing usage tracker for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:03:33.961260 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 5085dbf1-e506-4928-8a5c-fa72496efeb5 {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:03:33.968179 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-b76a81fe-e1be-40f5-973e-2481d13c2aff req-cb82ffc4-bbc0-48e5-96c6-78f70ee0b1d8 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Unset dirty status for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:03:33.969337 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b76a81fe-e1be-40f5-973e-2481d13c2aff req-cb82ffc4-bbc0-48e5-96c6-78f70ee0b1d8 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 931 time: 0.2807455 May 31 12:03:33.970927 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:34.004085 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Transition to ACTIVE for port object 5085dbf1-e506-4928-8a5c-fa72496efeb5 will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:03:34.024317 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b76a81fe-e1be-40f5-973e-2481d13c2aff req-8afb5f2d-b11f-4d9f-81bd-1f2385a56e6a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0492396 May 31 12:03:34.086033 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:34.090739 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-da385f42-4e7f-4d30-99b1-402e5a030870 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:34.252659 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Pushing event updated for resources: {'Port': ['ID=5085dbf1-e506-4928-8a5c-fa72496efeb5,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:34.279102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.188s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:34.321091 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Attempting to bind port 5085dbf1-e506-4928-8a5c-fa72496efeb5 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:03:34.321425 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Attempting to bind port 5085dbf1-e506-4928-8a5c-fa72496efeb5 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:03:34.321781 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Attempting to bind port 5085dbf1-e506-4928-8a5c-fa72496efeb5 on network ad7523f0-ba37-4688-b7be-179e0e8eae53 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:03:34.326664 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Port 5085dbf1-e506-4928-8a5c-fa72496efeb5 on network ad7523f0-ba37-4688-b7be-179e0e8eae53 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 12:03:34.326981 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Attempting to bind port 5085dbf1-e506-4928-8a5c-fa72496efeb5 on network ad7523f0-ba37-4688-b7be-179e0e8eae53 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:03:34.332066 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 3, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:03:34.332418 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Checking segment: {'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:03:34.332785 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Bound using segment: {'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:03:34.333205 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Bound port: 5085dbf1-e506-4928-8a5c-fa72496efeb5, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '865e26f1-d91f-476e-a356-62b51f1cbd7b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 382, 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:03:34.333552 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:34.371758 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd916d2c-166b-4866-bfd8-f671cabcd666 req-7d8bc0cf-5acc-42ac-a981-653257a37b16 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:34.373216 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bd916d2c-166b-4866-bfd8-f671cabcd666 req-7d8bc0cf-5acc-42ac-a981-653257a37b16 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 944 time: 0.0804470 May 31 12:03:34.420977 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] For port 5085dbf1-e506-4928-8a5c-fa72496efeb5, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:03:34.421348 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bd916d2c-166b-4866-bfd8-f671cabcd666 req-754dbb2f-fb71-4a5a-94ce-ffff07f9efc3 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0423467 May 31 12:03:34.424116 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] For port 5085dbf1-e506-4928-8a5c-fa72496efeb5, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5085dbf1-e506-4928-8a5c-fa72496efeb5,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:03:34.510182 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e4837294-fbd4-4d90-a6c0-e9caa29a5b28 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:03:34.529264 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:34.533089 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:34.594043 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:34.594696 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:34.595960 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] 10.222.0.47 "PUT /v2.0/ports/5085dbf1-e506-4928-8a5c-fa72496efeb5 HTTP/1.1" status: 200 len: 1093 time: 1.1611967 May 31 12:03:34.596448 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '55072c0c-2294-4520-886d-0beec11f9b50', 'tag': '5085dbf1-e506-4928-8a5c-fa72496efeb5'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:34.598768 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-c68c6f08-04e7-4c6d-adc6-48ed33856a1d" -d '{"events": [{"name": "network-changed", "server_uuid": "55072c0c-2294-4520-886d-0beec11f9b50", "tag": "5085dbf1-e506-4928-8a5c-fa72496efeb5"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:34.615529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Pushing event updated for resources: {'Port': ['ID=5085dbf1-e506-4928-8a5c-fa72496efeb5,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:34.618739 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:34.648856 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:03:34 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-8d5e6ba3-462c-481b-9d76-c6aa268f481a x-openstack-request-id: req-8d5e6ba3-462c-481b-9d76-c6aa268f481a {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:34.649180 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "55072c0c-2294-4520-886d-0beec11f9b50", "tag": "5085dbf1-e506-4928-8a5c-fa72496efeb5", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:34.649464 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-8d5e6ba3-462c-481b-9d76-c6aa268f481a {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:34.649936 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '55072c0c-2294-4520-886d-0beec11f9b50', 'tag': '5085dbf1-e506-4928-8a5c-fa72496efeb5', 'status': 'completed', 'code': 200} May 31 12:03:34.654238 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:03:34.659391 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:03:34.659771 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:34.740807 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c68c6f08-04e7-4c6d-adc6-48ed33856a1d req-a660da38-d761-466d-9a25-a462d5f6caab service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:34.742268 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c68c6f08-04e7-4c6d-adc6-48ed33856a1d req-a660da38-d761-466d-9a25-a462d5f6caab service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 1096 time: 0.0728490 May 31 12:03:34.776592 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:34.805934 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '7fd6c48b-9aad-4c92-bbef-75cc9871eb5c', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e4837294-fbd4-4d90-a6c0-e9caa29a5b28'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:34.808260 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-248fa5fa-2e98-46fa-8fb0-9139e2ce518f" -d '{"events": [{"server_uuid": "7fd6c48b-9aad-4c92-bbef-75cc9871eb5c", "name": "network-vif-plugged", "status": "completed", "tag": "e4837294-fbd4-4d90-a6c0-e9caa29a5b28"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:34.815597 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port e4837294-fbd4-4d90-a6c0-e9caa29a5b28, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:34.852320 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:03:34 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-ce8b524f-4975-4ea4-827d-a3f1997fce08 x-openstack-request-id: req-ce8b524f-4975-4ea4-827d-a3f1997fce08 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:34.852636 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "7fd6c48b-9aad-4c92-bbef-75cc9871eb5c", "name": "network-vif-plugged", "status": "completed", "tag": "e4837294-fbd4-4d90-a6c0-e9caa29a5b28", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:34.852888 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-ce8b524f-4975-4ea4-827d-a3f1997fce08 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:34.853322 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '7fd6c48b-9aad-4c92-bbef-75cc9871eb5c', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e4837294-fbd4-4d90-a6c0-e9caa29a5b28', 'code': 200} May 31 12:03:34.891130 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c68c6f08-04e7-4c6d-adc6-48ed33856a1d req-4d6d5cfc-c768-47d4-a1a8-3c5a15141f2c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:34.892704 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c68c6f08-04e7-4c6d-adc6-48ed33856a1d req-4d6d5cfc-c768-47d4-a1a8-3c5a15141f2c service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&shared=False HTTP/1.1" status: 200 len: 915 time: 0.1423588 May 31 12:03:34.897854 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:34.899855 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:34.968575 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:34.971696 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:35.116530 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c68c6f08-04e7-4c6d-adc6-48ed33856a1d req-3df17efc-4004-42b4-b49a-f5f55f3a71c6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:35.117992 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c68c6f08-04e7-4c6d-adc6-48ed33856a1d req-3df17efc-4004-42b4-b49a-f5f55f3a71c6 service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2194393 May 31 12:03:35.231971 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-8993477c-86bb-408e-82d3-3363d04a5e9c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:35.233661 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-8993477c-86bb-408e-82d3-3363d04a5e9c service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 1096 time: 0.0719299 May 31 12:03:35.276190 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-15717699-0285-46f4-87e2-c147cfb0a57c req-e5ea6888-8e13-482e-a4e4-bd2343f3ee1f tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:35.277643 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-15717699-0285-46f4-87e2-c147cfb0a57c req-e5ea6888-8e13-482e-a4e4-bd2343f3ee1f tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 943 time: 0.1360655 May 31 12:03:35.283391 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64ea4b8b-f245-42a3-bbee-2d56a22b7b6c req-c018e0cc-a13b-4b0d-a4c9-29d0b1470b0c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:35.284729 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-64ea4b8b-f245-42a3-bbee-2d56a22b7b6c req-c018e0cc-a13b-4b0d-a4c9-29d0b1470b0c service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 1098 time: 0.1084969 May 31 12:03:35.307815 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-223fa45e-472a-442c-9968-b54683bf3699 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=5085dbf1-e506-4928-8a5c-fa72496efeb5 HTTP/1.1" status: 200 len: 217 time: 0.0658796 May 31 12:03:35.350852 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-15717699-0285-46f4-87e2-c147cfb0a57c req-e3194f26-8292-4e43-9c7b-3774f2739299 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0651896 May 31 12:03:35.356467 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-64ea4b8b-f245-42a3-bbee-2d56a22b7b6c req-f30c2f32-810d-4756-b27d-b78e290927be service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0565279 May 31 12:03:35.373260 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e6e94ee9-e0a7-4a5c-8355-5dbef54e9f55 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:35.374709 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-e6e94ee9-e0a7-4a5c-8355-5dbef54e9f55 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=e4a37872-4de9-422e-a54c-71186b6f99d7 HTTP/1.1" status: 200 len: 854 time: 0.0603848 May 31 12:03:35.464722 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-0b4b97ec-9978-42c2-8605-52e642391644 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=ad7523f0-ba37-4688-b7be-179e0e8eae53&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0410526 May 31 12:03:35.655157 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-ef105fc9-c790-4992-9cd8-e7740378766d service neutron] 10.222.0.47 "GET /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1823068 May 31 12:03:35.689642 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c4cd0607-1cc6-446e-a416-6a918c7d1812 req-fe226033-7ff1-49d7-a68b-6758a0c53fe1 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:35.691012 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c4cd0607-1cc6-446e-a416-6a918c7d1812 req-fe226033-7ff1-49d7-a68b-6758a0c53fe1 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 945 time: 0.1475930 May 31 12:03:35.813227 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c4cd0607-1cc6-446e-a416-6a918c7d1812 req-55d8272f-f336-45c3-a3b9-cbe8c2b6f82a tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1165583 May 31 12:03:35.834243 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-5ed18961-7770-40e9-a11c-baeef023ad2f req-00ceff67-b029-4bb3-a0fd-228ae0a41983 service neutron] 10.222.0.47 "GET /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1717172 May 31 12:03:36.545222 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3de840a3-0431-4812-9dd5-eaa2f0455608 req-8ff35d98-fd28-4edd-954b-7bf30f0402f0 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:36.546671 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3de840a3-0431-4812-9dd5-eaa2f0455608 req-8ff35d98-fd28-4edd-954b-7bf30f0402f0 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 943 time: 0.0813684 May 31 12:03:36.591955 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3de840a3-0431-4812-9dd5-eaa2f0455608 req-bc517832-8bf2-45f5-9838-e7a155d87968 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0388906 May 31 12:03:37.000839 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d02a05a-ecc0-4525-84e3-d24c20028f60 req-da5cff3e-3061-4014-84fb-fb2944ad689f tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:37.002281 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1d02a05a-ecc0-4525-84e3-d24c20028f60 req-da5cff3e-3061-4014-84fb-fb2944ad689f tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 945 time: 0.0843277 May 31 12:03:37.046466 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1d02a05a-ecc0-4525-84e3-d24c20028f60 req-7685ac3f-4cc5-4665-a262-75e121916fcb tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384078 May 31 12:03:37.786024 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49f33a35-f9b7-41f0-900b-f8c3a187b9d3 req-0c1418c7-4669-4c9c-9d08-39cb48fb7d72 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:37.787468 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-49f33a35-f9b7-41f0-900b-f8c3a187b9d3 req-0c1418c7-4669-4c9c-9d08-39cb48fb7d72 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 943 time: 0.0869460 May 31 12:03:37.833061 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-49f33a35-f9b7-41f0-900b-f8c3a187b9d3 req-93201595-a046-40b7-9ed4-90b29f279d35 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0388896 May 31 12:03:38.518145 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5085dbf1-e506-4928-8a5c-fa72496efeb5 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:03:38.652364 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5085dbf1-e506-4928-8a5c-fa72496efeb5, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5085dbf1-e506-4928-8a5c-fa72496efeb5,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:38.665159 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '5085dbf1-e506-4928-8a5c-fa72496efeb5', 'network_id': 'ad7523f0-ba37-4688-b7be-179e0e8eae53', 'port_id': '5085dbf1-e506-4928-8a5c-fa72496efeb5', 'mac_address': 'fa:16:3e:82:0a:b5', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 382, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'e4a37872-4de9-422e-a54c-71186b6f99d7', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:03:38.720070 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:38.740428 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 5085dbf1-e506-4928-8a5c-fa72496efeb5 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:03:38.756259 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5085dbf1-e506-4928-8a5c-fa72496efeb5, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5085dbf1-e506-4928-8a5c-fa72496efeb5,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:38.820353 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:38.824003 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:38.856491 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2390f9a6-f722-44ea-87a0-308a95386b55 req-6f41f92e-e64c-426f-8d5b-a5ed6f0e46dd tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:38.858926 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2390f9a6-f722-44ea-87a0-308a95386b55 req-6f41f92e-e64c-426f-8d5b-a5ed6f0e46dd tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 945 time: 0.0797522 May 31 12:03:38.891244 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=5085dbf1-e506-4928-8a5c-fa72496efeb5,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:38.894352 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:38.904162 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2390f9a6-f722-44ea-87a0-308a95386b55 req-9db67a71-d643-4c62-80ff-df9e9baed652 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0402834 May 31 12:03:39.036684 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d665956-fde9-47dc-b68d-a1fe0ea9abda req-2aaf6f43-5bba-42cd-b8e4-89062b52e155 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:39.038160 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0d665956-fde9-47dc-b68d-a1fe0ea9abda req-2aaf6f43-5bba-42cd-b8e4-89062b52e155 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 944 time: 0.0832863 May 31 12:03:39.055203 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['5085dbf1-e506-4928-8a5c-fa72496efeb5'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:03:39.083630 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0d665956-fde9-47dc-b68d-a1fe0ea9abda req-2bc7250a-432c-4f0c-9712-3f29bf7f4d01 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390029 May 31 12:03:39.419768 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:40.106921 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-546ac2dc-f273-4001-aa39-dc9e28957225 req-242b1647-0e5d-4ad5-94f1-03509278cd94 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:40.109224 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-546ac2dc-f273-4001-aa39-dc9e28957225 req-242b1647-0e5d-4ad5-94f1-03509278cd94 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 945 time: 0.0909603 May 31 12:03:40.165015 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-546ac2dc-f273-4001-aa39-dc9e28957225 req-519f5114-123f-4b8d-bd3f-e558b565d216 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0487893 May 31 12:03:40.375266 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8981759c-eb26-49a5-8751-0df427f79f63 req-b1f416f9-e44d-4b96-8ab5-6d10f91b72ea tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:40.377316 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8981759c-eb26-49a5-8751-0df427f79f63 req-b1f416f9-e44d-4b96-8ab5-6d10f91b72ea tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.47 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 945 time: 0.2827206 May 31 12:03:40.438896 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8759a6f-6684-442a-9efc-4b04626fa0ee req-2231df06-292a-465f-a8d2-10ce9d5f7c1c tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:40.440399 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c8759a6f-6684-442a-9efc-4b04626fa0ee req-2231df06-292a-465f-a8d2-10ce9d5f7c1c tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 944 time: 0.1866977 May 31 12:03:40.529020 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5085dbf1-e506-4928-8a5c-fa72496efeb5 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:03:40.560819 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c8759a6f-6684-442a-9efc-4b04626fa0ee req-5e6ec6bc-988c-4509-ba8f-0ceb1aff4e1b tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1111424 May 31 12:03:40.593951 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Deleting port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:03:40.594309 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:40.683132 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 5085dbf1-e506-4928-8a5c-fa72496efeb5 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:03:40.688482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 5085dbf1-e506-4928-8a5c-fa72496efeb5 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:03:40.688883 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:40.752905 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] For port e4837294-fbd4-4d90-a6c0-e9caa29a5b28, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:40.753266 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:40.820998 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:40.848987 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '55072c0c-2294-4520-886d-0beec11f9b50', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5085dbf1-e506-4928-8a5c-fa72496efeb5'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:40.851240 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-22051512-c1bb-4a5b-9eee-9b5badfc3178" -d '{"events": [{"server_uuid": "55072c0c-2294-4520-886d-0beec11f9b50", "name": "network-vif-plugged", "status": "completed", "tag": "5085dbf1-e506-4928-8a5c-fa72496efeb5"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:40.860615 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 5085dbf1-e506-4928-8a5c-fa72496efeb5, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5085dbf1-e506-4928-8a5c-fa72496efeb5,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:40.892860 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Calling delete_port for e4837294-fbd4-4d90-a6c0-e9caa29a5b28 owned by compute:nova {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:03:40.894585 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:03:40 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-1c14042d-377d-4689-add7-942879b7c9af x-openstack-request-id: req-1c14042d-377d-4689-add7-942879b7c9af {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:40.894890 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "55072c0c-2294-4520-886d-0beec11f9b50", "name": "network-vif-plugged", "status": "completed", "tag": "5085dbf1-e506-4928-8a5c-fa72496efeb5", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:40.895159 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-1c14042d-377d-4689-add7-942879b7c9af {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:40.895562 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '55072c0c-2294-4520-886d-0beec11f9b50', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5085dbf1-e506-4928-8a5c-fa72496efeb5', 'code': 200} May 31 12:03:40.926091 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:40.927929 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:40.993361 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=5085dbf1-e506-4928-8a5c-fa72496efeb5,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:40.996326 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:41.077626 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:41.084823 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7fb9afb1-07f3-4a41-9aeb-d11f01cb308e tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:41.090362 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['e4837294-fbd4-4d90-a6c0-e9caa29a5b28'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:03:41.163580 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Pushing event deleted for resources: {'Port': ['ID=e4837294-fbd4-4d90-a6c0-e9caa29a5b28,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:41.166318 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:41.212329 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] No DVR routers for this DVR port e4837294-fbd4-4d90-a6c0-e9caa29a5b28 on host hv-cinder-79123 {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:03:41.215678 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:41.218552 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '7fd6c48b-9aad-4c92-bbef-75cc9871eb5c', 'name': 'network-vif-deleted', 'tag': 'e4837294-fbd4-4d90-a6c0-e9caa29a5b28'}] {{(pid=99039) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:41.220799 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}e91b859d0027e85cebf6c806e8859e3c7d2b43c4c06023ea3cec25c1ab046ca5" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-e6b35721-28f5-4858-a09f-e94d8cce21f9" -d '{"events": [{"server_uuid": "7fd6c48b-9aad-4c92-bbef-75cc9871eb5c", "name": "network-vif-deleted", "tag": "e4837294-fbd4-4d90-a6c0-e9caa29a5b28"}]}' {{(pid=99039) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:41.267212 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:03:41 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-3e901b4b-e004-49c9-8018-b116a8cd03d6 x-openstack-request-id: req-3e901b4b-e004-49c9-8018-b116a8cd03d6 {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:41.267603 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "7fd6c48b-9aad-4c92-bbef-75cc9871eb5c", "name": "network-vif-deleted", "tag": "e4837294-fbd4-4d90-a6c0-e9caa29a5b28", "status": "completed", "code": 200}]} {{(pid=99039) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:41.267956 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-3e901b4b-e004-49c9-8018-b116a8cd03d6 {{(pid=99039) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:41.268469 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '7fd6c48b-9aad-4c92-bbef-75cc9871eb5c', 'name': 'network-vif-deleted', 'tag': 'e4837294-fbd4-4d90-a6c0-e9caa29a5b28', 'status': 'completed', 'code': 200} May 31 12:03:41.323685 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-206382ab-2ce2-497e-8f3b-17f23fe84ac3 req-d2688adc-2f3a-472e-94f9-6206b5ca8cfd tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Synchronizing usage tracker for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:port {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:03:41.328257 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8981759c-eb26-49a5-8751-0df427f79f63 req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.47 "DELETE /v2.0/ports/e4837294-fbd4-4d90-a6c0-e9caa29a5b28 HTTP/1.1" status: 204 len: 173 time: 0.9440279 May 31 12:03:41.372406 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52bf6bf1-462f-42d7-bbbc-222b03be35c7 req-b5689222-aff7-47a7-9e5f-29ed2545cb8b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:41.373901 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-52bf6bf1-462f-42d7-bbbc-222b03be35c7 req-b5689222-aff7-47a7-9e5f-29ed2545cb8b service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 1098 time: 0.0717487 May 31 12:03:41.423382 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-206382ab-2ce2-497e-8f3b-17f23fe84ac3 req-d2688adc-2f3a-472e-94f9-6206b5ca8cfd tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Unset dirty status for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:port {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:03:41.424030 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-52bf6bf1-462f-42d7-bbbc-222b03be35c7 req-6c484607-806b-4f20-8c20-6ac873ded4db service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0373662 May 31 12:03:41.425123 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-206382ab-2ce2-497e-8f3b-17f23fe84ac3 req-d2688adc-2f3a-472e-94f9-6206b5ca8cfd tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=7fd6c48b-9aad-4c92-bbef-75cc9871eb5c HTTP/1.1" status: 200 len: 210 time: 0.1500237 May 31 12:03:41.750656 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08aad2f5-bec0-4b26-adda-f895b2761fff req-737c3611-c321-4ef4-91a8-943b911ea12f tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:41.752098 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-08aad2f5-bec0-4b26-adda-f895b2761fff req-737c3611-c321-4ef4-91a8-943b911ea12f tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 945 time: 0.0802810 May 31 12:03:41.795305 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-08aad2f5-bec0-4b26-adda-f895b2761fff req-2cf87f3a-9c70-4e16-a760-7a05113a21a0 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0377111 May 31 12:03:42.794473 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-351f511b-7cfb-4256-b823-c851a149f40d req-50c3099c-3ae3-4f5f-93ab-4fff99b981e8 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:42.795880 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-50c3099c-3ae3-4f5f-93ab-4fff99b981e8 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/networks?id=1b0e8409-da86-4bc0-b712-3072314a6793 HTTP/1.1" status: 200 len: 808 time: 0.1461279 May 31 12:03:42.810727 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-8c7dd6b7-ad03-4c79-af7e-99b925f2b6cd tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/quotas/e946adaaf3ee413da7e5acaa60376c5b HTTP/1.1" status: 200 len: 375 time: 0.0093777 May 31 12:03:42.861312 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-e742bf49-a966-4175-b723-ddd1f23a6287 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0457602 May 31 12:03:42.987976 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-daab30a7-e3ec-42a4-911b-98d6b9ff58ec req-239df774-e202-4579-a148-21ec3ce400d9 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:42.989437 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-daab30a7-e3ec-42a4-911b-98d6b9ff58ec req-239df774-e202-4579-a148-21ec3ce400d9 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 945 time: 0.0851822 May 31 12:03:42.997139 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-c1f1b3a3-045c-47d3-b613-0501eda9d76b service neutron] 10.222.0.48 "GET /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1287427 May 31 12:03:43.034802 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-daab30a7-e3ec-42a4-911b-98d6b9ff58ec req-a3f404ce-ef01-4271-97df-a919931169de tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393937 May 31 12:03:43.129419 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-7e07e25b-8134-4fc2-b22c-3a160a1b1cba service neutron] 10.222.0.48 "GET /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1265700 May 31 12:03:43.411405 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1f163bd7-d829-4dd8-9b7a-4678f932fa32 req-20116009-1202-4fe8-a2cb-2eb092e0bb24 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 210 time: 0.0448678 May 31 12:03:44.529633 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e4837294-fbd4-4d90-a6c0-e9caa29a5b28 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:03:44.561406 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8b32e289-a67e-4500-ba98-8902f057db01 req-157afce4-2396-4c16-a042-8b65ec9a9f4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 210 time: 0.0430844 May 31 12:03:44.564508 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: e4837294-fbd4-4d90-a6c0-e9caa29a5b28 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:03:44.564859 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device e4837294-fbd4-4d90-a6c0-e9caa29a5b28 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:03:45.711940 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-05099ab8-d41b-4f43-9d36-6dbbf540d3b1 req-f224f78f-57f8-4147-a977-a0f053713681 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 210 time: 0.0420947 May 31 12:03:46.178224 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-351f511b-7cfb-4256-b823-c851a149f40d req-710b9ea5-590f-4c9d-b56b-2548e1d2e74b tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:46.179613 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-710b9ea5-590f-4c9d-b56b-2548e1d2e74b tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.47 "GET /v2.0/networks?id=1b0e8409-da86-4bc0-b712-3072314a6793 HTTP/1.1" status: 200 len: 808 time: 0.1489232 May 31 12:03:46.192849 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Request body: {'port': {'device_id': '2a9190dc-89fc-4c74-b303-57306e92caf5', 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793', 'admin_state_up': True, 'tenant_id': 'e946adaaf3ee413da7e5acaa60376c5b'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:03:46.414076 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:03:46.424101 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:03:46.511756 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Made reservation on behalf of e946adaaf3ee413da7e5acaa60376c5b for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:03:46.515558 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:46.678372 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Allocated IP 172.20.0.8 (1b0e8409-da86-4bc0-b712-3072314a6793/1d9b77a2-8a70-4491-974c-8c516e56cff2/fb6f58a9-55da-462e-9acb-67f1c18de634) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:03:46.869625 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-46c1f0b1-a0a4-4c2a-abda-23a51b58fb65 req-2b7be980-9819-41d4-9fd7-f0465b0b8bd6 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 210 time: 0.0491686 May 31 12:03:46.918402 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:47.029527 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:47.035291 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:47.099241 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Network 1b0e8409-da86-4bc0-b712-3072314a6793 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:03:47.102338 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:03:47.107426 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:03:47.147535 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Pushing event updated for resources: {'Port': ['ID=fb6f58a9-55da-462e-9acb-67f1c18de634,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:47.152911 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:47.183746 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:47.184479 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:47.294971 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.1055565 May 31 12:03:47.307571 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Request body: {'port': {'device_id': '2a9190dc-89fc-4c74-b303-57306e92caf5', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:03:47.418672 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:47.752716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id fb6f58a9-55da-462e-9acb-67f1c18de634 {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:03:47.759755 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:47.773502 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Transition to ACTIVE for port object fb6f58a9-55da-462e-9acb-67f1c18de634 will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:03:47.846028 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:47.849717 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2206d3b3-69ca-4599-afea-583ca7ab3515 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:48.004614 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Pushing event updated for resources: {'Port': ['ID=fb6f58a9-55da-462e-9acb-67f1c18de634,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:48.011687 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:48.070251 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a85ed63c-44a2-421b-9763-34e61cfe13c5 req-1f0c740d-491d-419c-9616-e5073b082083 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:48.071743 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a85ed63c-44a2-421b-9763-34e61cfe13c5 req-1f0c740d-491d-419c-9616-e5073b082083 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 943 time: 0.0908177 May 31 12:03:48.081402 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Attempting to bind port fb6f58a9-55da-462e-9acb-67f1c18de634 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:03:48.081767 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Attempting to bind port fb6f58a9-55da-462e-9acb-67f1c18de634 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:03:48.082093 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Attempting to bind port fb6f58a9-55da-462e-9acb-67f1c18de634 on network 1b0e8409-da86-4bc0-b712-3072314a6793 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:03:48.086808 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Port fb6f58a9-55da-462e-9acb-67f1c18de634 on network 1b0e8409-da86-4bc0-b712-3072314a6793 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 12:03:48.087143 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Attempting to bind port fb6f58a9-55da-462e-9acb-67f1c18de634 on network 1b0e8409-da86-4bc0-b712-3072314a6793 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:03:48.093062 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 3, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:03:48.098694 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Checking segment: {'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:03:48.098694 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Bound using segment: {'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:03:48.098694 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Bound port: fb6f58a9-55da-462e-9acb-67f1c18de634, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 354, 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:03:48.098694 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:48.118097 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a85ed63c-44a2-421b-9763-34e61cfe13c5 req-a62fd4a1-d35f-4580-a3f2-d9f89cc6febb tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0405464 May 31 12:03:48.185966 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] For port fb6f58a9-55da-462e-9acb-67f1c18de634, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:03:48.188869 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] For port fb6f58a9-55da-462e-9acb-67f1c18de634, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb6f58a9-55da-462e-9acb-67f1c18de634,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:03:48.275864 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:48.280362 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7b2d6b7e-8dd2-4995-aaaf-7d162e46ddbf service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:48.331249 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:48.331885 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:48.333254 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] 10.222.0.47 "PUT /v2.0/ports/fb6f58a9-55da-462e-9acb-67f1c18de634 HTTP/1.1" status: 200 len: 1093 time: 1.0302303 May 31 12:03:48.341005 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '2a9190dc-89fc-4c74-b303-57306e92caf5', 'tag': 'fb6f58a9-55da-462e-9acb-67f1c18de634'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:48.343292 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-0c10f392-6322-4337-a34d-4223ef5eab2e" -d '{"events": [{"name": "network-changed", "server_uuid": "2a9190dc-89fc-4c74-b303-57306e92caf5", "tag": "fb6f58a9-55da-462e-9acb-67f1c18de634"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:48.365061 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Pushing event updated for resources: {'Port': ['ID=fb6f58a9-55da-462e-9acb-67f1c18de634,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:48.368699 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:48.384806 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:03:48 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-fecda33c-1ae4-4a8a-ac52-47a7fe7a2e78 x-openstack-request-id: req-fecda33c-1ae4-4a8a-ac52-47a7fe7a2e78 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:48.385153 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "2a9190dc-89fc-4c74-b303-57306e92caf5", "tag": "fb6f58a9-55da-462e-9acb-67f1c18de634", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:48.385439 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-fecda33c-1ae4-4a8a-ac52-47a7fe7a2e78 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:48.385891 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '2a9190dc-89fc-4c74-b303-57306e92caf5', 'tag': 'fb6f58a9-55da-462e-9acb-67f1c18de634', 'status': 'completed', 'code': 200} May 31 12:03:48.475081 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c10f392-6322-4337-a34d-4223ef5eab2e req-e5a821a4-de24-4ded-9b4d-aae952ce6a20 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:48.476562 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0c10f392-6322-4337-a34d-4223ef5eab2e req-e5a821a4-de24-4ded-9b4d-aae952ce6a20 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 1096 time: 0.0709260 May 31 12:03:48.701175 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c10f392-6322-4337-a34d-4223ef5eab2e req-c8281914-9e65-459d-80e7-fa2a02defe78 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:48.702710 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0c10f392-6322-4337-a34d-4223ef5eab2e req-c8281914-9e65-459d-80e7-fa2a02defe78 service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&shared=False HTTP/1.1" status: 200 len: 909 time: 0.2190454 May 31 12:03:48.845991 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c10f392-6322-4337-a34d-4223ef5eab2e req-73402904-e9f5-49ad-84b8-3e2063a13d6a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:48.847413 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0c10f392-6322-4337-a34d-4223ef5eab2e req-73402904-e9f5-49ad-84b8-3e2063a13d6a service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1379802 May 31 12:03:48.962111 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-351f511b-7cfb-4256-b823-c851a149f40d req-2c3f1940-93cb-4855-bf69-cbf0bf693d22 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:48.965557 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-2c3f1940-93cb-4855-bf69-cbf0bf693d22 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 1096 time: 0.0820155 May 31 12:03:49.039607 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-7f4f1159-2d85-42fc-95af-abc0b382c422 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=fb6f58a9-55da-462e-9acb-67f1c18de634 HTTP/1.1" status: 200 len: 217 time: 0.0622401 May 31 12:03:49.110900 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-351f511b-7cfb-4256-b823-c851a149f40d req-8e107b4c-9b4d-4c26-83af-b1079d3393a7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:49.110900 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-8e107b4c-9b4d-4c26-83af-b1079d3393a7 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=1d9b77a2-8a70-4491-974c-8c516e56cff2 HTTP/1.1" status: 200 len: 848 time: 0.0621927 May 31 12:03:49.159845 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-68a22043-590d-4783-959a-0f738c99cc0e service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=1b0e8409-da86-4bc0-b712-3072314a6793&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0444403 May 31 12:03:49.302040 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-932b252a-d4f4-4a3f-952f-fc484f6017fc service neutron] 10.222.0.47 "GET /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1350393 May 31 12:03:49.435202 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-351f511b-7cfb-4256-b823-c851a149f40d req-295dcf48-8b2b-43ae-975b-f6a216d901a0 service neutron] 10.222.0.47 "GET /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1263831 May 31 12:03:49.457338 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1fd87e8a-af98-4ce2-bdbd-b64e2d6979ec req-050a0045-677a-4400-bbfc-ac3e2c0a21f2 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:49.458792 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1fd87e8a-af98-4ce2-bdbd-b64e2d6979ec req-050a0045-677a-4400-bbfc-ac3e2c0a21f2 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 943 time: 0.0794604 May 31 12:03:49.508021 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1fd87e8a-af98-4ce2-bdbd-b64e2d6979ec req-8deb6b68-f766-4a04-960d-6c2cc919f55a tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0426822 May 31 12:03:50.709269 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ace7f3a3-b89b-4858-b979-70b1fef04e06 req-c55a7425-eabf-494d-a3a1-c71fa1918e03 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:50.709998 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-ace7f3a3-b89b-4858-b979-70b1fef04e06 req-c55a7425-eabf-494d-a3a1-c71fa1918e03 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Synchronizing usage tracker for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:03:50.785475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-ace7f3a3-b89b-4858-b979-70b1fef04e06 req-c55a7425-eabf-494d-a3a1-c71fa1918e03 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Unset dirty status for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:03:50.786663 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ace7f3a3-b89b-4858-b979-70b1fef04e06 req-c55a7425-eabf-494d-a3a1-c71fa1918e03 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 943 time: 0.1738913 May 31 12:03:50.833186 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ace7f3a3-b89b-4858-b979-70b1fef04e06 req-2faaa3db-8801-42e0-a519-68604b0844ed tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0402441 May 31 12:03:50.865488 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e2d759f2-a20b-4719-9e5c-8fb8ec55a62a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:52.015449 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7634619b-da6d-4560-9059-e710f0613efc req-3be8603b-d4b4-402a-a849-6b764a3bbc52 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:52.016871 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7634619b-da6d-4560-9059-e710f0613efc req-3be8603b-d4b4-402a-a849-6b764a3bbc52 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 943 time: 0.0799096 May 31 12:03:52.061999 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7634619b-da6d-4560-9059-e710f0613efc req-a3b8450e-04a4-46e9-bfc0-6dded14700b4 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0392461 May 31 12:03:52.578564 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device fb6f58a9-55da-462e-9acb-67f1c18de634 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:03:52.708562 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port fb6f58a9-55da-462e-9acb-67f1c18de634, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb6f58a9-55da-462e-9acb-67f1c18de634,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:52.721800 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'fb6f58a9-55da-462e-9acb-67f1c18de634', 'network_id': '1b0e8409-da86-4bc0-b712-3072314a6793', 'port_id': 'fb6f58a9-55da-462e-9acb-67f1c18de634', 'mac_address': 'fa:16:3e:38:8a:92', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 354, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1d9b77a2-8a70-4491-974c-8c516e56cff2', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:03:52.778593 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:52.795546 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id fb6f58a9-55da-462e-9acb-67f1c18de634 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:03:52.810380 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port fb6f58a9-55da-462e-9acb-67f1c18de634, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb6f58a9-55da-462e-9acb-67f1c18de634,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:52.893815 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:52.896331 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-bfb123b4-0191-4f6b-b8aa-f27c9b2f48a3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:52.961232 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=fb6f58a9-55da-462e-9acb-67f1c18de634,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:52.965094 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:53.246808 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-419d55b1-13cf-4db4-ab6a-a823763d846d req-f1d8c60c-522d-4309-9100-cab9e7130133 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:53.248310 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-419d55b1-13cf-4db4-ab6a-a823763d846d req-f1d8c60c-522d-4309-9100-cab9e7130133 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 944 time: 0.0820777 May 31 12:03:53.297333 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-419d55b1-13cf-4db4-ab6a-a823763d846d req-f42cfaa1-36af-4ab2-ac27-0a2f6079b4bb tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0434833 May 31 12:03:53.348758 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:53.445923 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-22dfc1f9-23d2-437c-93ab-025596e4e44f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:54.465089 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:03:54.493237 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e9b5324-33a2-4c73-9a26-3a500b2d1c13 req-1eadead3-c717-408d-824d-aa0bf6256842 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:54.494728 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4e9b5324-33a2-4c73-9a26-3a500b2d1c13 req-1eadead3-c717-408d-824d-aa0bf6256842 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 944 time: 0.0867355 May 31 12:03:54.539295 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4e9b5324-33a2-4c73-9a26-3a500b2d1c13 req-252f08d6-641d-41ff-9e72-fd225612234d tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389578 May 31 12:03:54.613919 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:55.726935 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3e99dcc3-7974-408a-8610-9b56318d501d req-43a42d1e-663b-44d3-947b-4e8c63eeef62 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:55.728406 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3e99dcc3-7974-408a-8610-9b56318d501d req-43a42d1e-663b-44d3-947b-4e8c63eeef62 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 944 time: 0.0832810 May 31 12:03:55.772439 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3e99dcc3-7974-408a-8610-9b56318d501d req-3cd6b0b6-ca4e-4dc2-83ac-507dc05aaf57 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384002 May 31 12:03:55.910783 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['fb6f58a9-55da-462e-9acb-67f1c18de634'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:03:56.154595 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:56.957970 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2389086e-dc29-4300-aafd-8908a84fc106 req-177d77a9-17bc-4b58-be1c-ff065415a4ec tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:56.959306 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2389086e-dc29-4300-aafd-8908a84fc106 req-177d77a9-17bc-4b58-be1c-ff065415a4ec tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 944 time: 0.0841789 May 31 12:03:57.004273 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-2389086e-dc29-4300-aafd-8908a84fc106 req-41035e43-ddde-4ed8-a794-eb0fdc3c09be tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0387750 May 31 12:03:58.189376 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f1f5bd2-6c8a-4282-995c-65a95133917e req-cd326724-c84c-49bd-965e-49860094ccdb tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:58.190834 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f1f5bd2-6c8a-4282-995c-65a95133917e req-cd326724-c84c-49bd-965e-49860094ccdb tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 944 time: 0.0803084 May 31 12:03:58.237888 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-4f1f5bd2-6c8a-4282-995c-65a95133917e req-f97b33fe-1b0f-4917-9d4e-555563abaa14 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0412705 May 31 12:03:58.261637 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device fb6f58a9-55da-462e-9acb-67f1c18de634 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:03:58.402074 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port fb6f58a9-55da-462e-9acb-67f1c18de634 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:03:58.406902 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port fb6f58a9-55da-462e-9acb-67f1c18de634 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:03:58.411232 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:58.522542 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:58.549458 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '2a9190dc-89fc-4c74-b303-57306e92caf5', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'fb6f58a9-55da-462e-9acb-67f1c18de634'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:03:58.551649 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-d54a1df1-7696-4d04-8699-7e87f8bd7c60" -d '{"events": [{"server_uuid": "2a9190dc-89fc-4c74-b303-57306e92caf5", "name": "network-vif-plugged", "status": "completed", "tag": "fb6f58a9-55da-462e-9acb-67f1c18de634"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:03:58.559744 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port fb6f58a9-55da-462e-9acb-67f1c18de634, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb6f58a9-55da-462e-9acb-67f1c18de634,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:03:58.592231 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:03:58 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-cb22c25e-96a4-4d4b-a775-0f6036461424 x-openstack-request-id: req-cb22c25e-96a4-4d4b-a775-0f6036461424 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:03:58.592536 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "2a9190dc-89fc-4c74-b303-57306e92caf5", "name": "network-vif-plugged", "status": "completed", "tag": "fb6f58a9-55da-462e-9acb-67f1c18de634", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:03:58.592793 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-cb22c25e-96a4-4d4b-a775-0f6036461424 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:03:58.593186 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '2a9190dc-89fc-4c74-b303-57306e92caf5', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'fb6f58a9-55da-462e-9acb-67f1c18de634', 'code': 200} May 31 12:03:58.642143 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:03:58.643996 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:03:58.717043 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=fb6f58a9-55da-462e-9acb-67f1c18de634,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:03:58.720047 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:03:59.418857 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6920350c-3146-4b74-bd2f-fb73c648cbb9 req-99864d16-9625-41f3-a5f4-76c1b94fdc51 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:03:59.420275 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6920350c-3146-4b74-bd2f-fb73c648cbb9 req-99864d16-9625-41f3-a5f4-76c1b94fdc51 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 945 time: 0.0791020 May 31 12:03:59.464290 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6920350c-3146-4b74-bd2f-fb73c648cbb9 req-03853337-d04c-4853-b17a-98d4d9159761 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384367 May 31 12:04:00.543216 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ef496ccd-a0b1-4ae7-936b-fd479ef412fe req-8c3fa756-9ddd-4f2a-9300-397e2087bc6d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:00.544735 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ef496ccd-a0b1-4ae7-936b-fd479ef412fe req-8c3fa756-9ddd-4f2a-9300-397e2087bc6d service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 1098 time: 0.0761025 May 31 12:04:00.586649 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ef496ccd-a0b1-4ae7-936b-fd479ef412fe req-5ce8f9c9-1726-4fe5-9fb6-5118abdaaeff service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0347197 May 31 12:04:00.653954 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7890649d-c57c-4c76-99f8-99adeeaf96eb req-3729529b-0ad4-4799-a063-3ff2149fd116 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:00.655394 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7890649d-c57c-4c76-99f8-99adeeaf96eb req-3729529b-0ad4-4799-a063-3ff2149fd116 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 945 time: 0.0832713 May 31 12:04:00.698730 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-7890649d-c57c-4c76-99f8-99adeeaf96eb req-201d2b94-a8d2-44f8-9c04-38a6f0d22105 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0376794 May 31 12:04:01.882881 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7524cde-d48d-441f-a570-632eeef09c5b req-b2198f95-350f-4bd9-bfbc-a8c748f45e73 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:01.884837 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7524cde-d48d-441f-a570-632eeef09c5b req-b2198f95-350f-4bd9-bfbc-a8c748f45e73 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 945 time: 0.0791147 May 31 12:04:01.928784 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7524cde-d48d-441f-a570-632eeef09c5b req-81e9df0b-0546-4735-9527-5dc627ffbc4b tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0381846 May 31 12:04:03.763970 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2c2916e-bd27-48d0-8625-cd3ae9a33276 req-976d0629-58c8-473c-ba5a-320943935794 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:03.765435 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d2c2916e-bd27-48d0-8625-cd3ae9a33276 req-976d0629-58c8-473c-ba5a-320943935794 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 945 time: 0.0849798 May 31 12:04:03.811014 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-d2c2916e-bd27-48d0-8625-cd3ae9a33276 req-f93de1be-dbe8-480e-9f92-a24864be871e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396554 May 31 12:04:04.994799 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb0d0c53-7bdb-462f-88bf-84436da6cf4c req-ef300e1d-aee4-4c3e-912b-0eac12ba1087 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:04.996283 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-eb0d0c53-7bdb-462f-88bf-84436da6cf4c req-ef300e1d-aee4-4c3e-912b-0eac12ba1087 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 945 time: 0.0800791 May 31 12:04:05.048295 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-eb0d0c53-7bdb-462f-88bf-84436da6cf4c req-f34f807b-16e8-4982-8f87-dd6428ccae75 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/security-groups?id=2e80e113-7a5f-431b-b1bc-fdc03edd754f&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0446351 May 31 12:04:05.089262 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-500593d7-0a7a-4d5f-9443-f77b677a80de tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:05.090826 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-500593d7-0a7a-4d5f-9443-f77b677a80de tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.47 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 945 time: 0.0973606 May 31 12:04:05.303839 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Deleting port fb6f58a9-55da-462e-9acb-67f1c18de634 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:05.304102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:05.438340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] For port fb6f58a9-55da-462e-9acb-67f1c18de634, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb6f58a9-55da-462e-9acb-67f1c18de634,segment=NetworkSegment(696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90),segment_id=696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:05.438695 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:05.568894 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Calling delete_port for fb6f58a9-55da-462e-9acb-67f1c18de634 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:05.771612 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:05.777304 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6faf21ff-bba3-4353-885d-84c1d8ec8ad9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:05.821317 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Pushing event deleted for resources: {'Port': ['ID=fb6f58a9-55da-462e-9acb-67f1c18de634,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:05.824340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:05.897529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] No DVR routers for this DVR port fb6f58a9-55da-462e-9acb-67f1c18de634 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:04:05.900380 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:05.903019 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '2a9190dc-89fc-4c74-b303-57306e92caf5', 'name': 'network-vif-deleted', 'tag': 'fb6f58a9-55da-462e-9acb-67f1c18de634'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:04:05.905268 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-6cd8d5e5-b7c8-44ce-8bc9-aaa4f060e6ab" -d '{"events": [{"server_uuid": "2a9190dc-89fc-4c74-b303-57306e92caf5", "name": "network-vif-deleted", "tag": "fb6f58a9-55da-462e-9acb-67f1c18de634"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:04:05.949634 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:04:05 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-0da98f0e-e362-4d5a-8823-93779290ec20 x-openstack-request-id: req-0da98f0e-e362-4d5a-8823-93779290ec20 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:04:05.949634 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "2a9190dc-89fc-4c74-b303-57306e92caf5", "name": "network-vif-deleted", "tag": "fb6f58a9-55da-462e-9acb-67f1c18de634", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:04:05.949634 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-0da98f0e-e362-4d5a-8823-93779290ec20 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:04:05.949985 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '2a9190dc-89fc-4c74-b303-57306e92caf5', 'name': 'network-vif-deleted', 'tag': 'fb6f58a9-55da-462e-9acb-67f1c18de634', 'status': 'completed', 'code': 200} May 31 12:04:06.035395 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8f27d226-712c-44cc-af73-75d0bfd50b69 req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.47 "DELETE /v2.0/ports/fb6f58a9-55da-462e-9acb-67f1c18de634 HTTP/1.1" status: 204 len: 173 time: 0.9379480 May 31 12:04:06.344751 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-8bd108f5-3dcb-46fb-a72e-f14c98e6b560 req-fb076aba-972a-462d-ba5e-ae7610ff320a tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Synchronizing usage tracker for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:04:06.425059 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-8bd108f5-3dcb-46fb-a72e-f14c98e6b560 req-fb076aba-972a-462d-ba5e-ae7610ff320a tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Unset dirty status for tenant:e946adaaf3ee413da7e5acaa60376c5b on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:04:06.426394 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8bd108f5-3dcb-46fb-a72e-f14c98e6b560 req-fb076aba-972a-462d-ba5e-ae7610ff320a tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] 10.222.0.48 "GET /v2.0/ports?device_id=2a9190dc-89fc-4c74-b303-57306e92caf5 HTTP/1.1" status: 200 len: 210 time: 0.1269217 May 31 12:04:07.159037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbc8f765-14e5-4453-852b-829249ad7631 req-d16688f8-a3a7-4488-9d3d-c358f1fdaa80 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:07.160470 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dbc8f765-14e5-4453-852b-829249ad7631 req-d16688f8-a3a7-4488-9d3d-c358f1fdaa80 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 945 time: 0.0842004 May 31 12:04:07.204228 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dbc8f765-14e5-4453-852b-829249ad7631 req-e1a1f5c1-4cef-40cf-ab00-d2c74a6a15bd tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0374141 May 31 12:04:07.675096 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device fb6f58a9-55da-462e-9acb-67f1c18de634 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:04:07.708913 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: fb6f58a9-55da-462e-9acb-67f1c18de634 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:04:07.709202 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device fb6f58a9-55da-462e-9acb-67f1c18de634 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:04:08.389654 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc87a3cd-9477-4a71-a491-d8446642d624 req-1623b91f-2e8d-4506-87c6-bec4e001c7eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:08.391083 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc87a3cd-9477-4a71-a491-d8446642d624 req-1623b91f-2e8d-4506-87c6-bec4e001c7eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 945 time: 0.0799501 May 31 12:04:08.440298 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-dc87a3cd-9477-4a71-a491-d8446642d624 req-08df115e-5332-494c-9097-18f3b6c1068a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/security-groups?id=48250270-c82c-483c-a3cc-9d463d413e15&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0436027 May 31 12:04:08.692758 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-8664dff6-a83f-47d5-ada6-0a1367124346 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:08.694198 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-8664dff6-a83f-47d5-ada6-0a1367124346 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.47 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 945 time: 0.0800543 May 31 12:04:08.834675 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Deleting port 5085dbf1-e506-4928-8a5c-fa72496efeb5 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:08.835015 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:08.967971 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] For port 5085dbf1-e506-4928-8a5c-fa72496efeb5, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5085dbf1-e506-4928-8a5c-fa72496efeb5,segment=NetworkSegment(865e26f1-d91f-476e-a356-62b51f1cbd7b),segment_id=865e26f1-d91f-476e-a356-62b51f1cbd7b)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:08.968325 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:09.096804 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Calling delete_port for 5085dbf1-e506-4928-8a5c-fa72496efeb5 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:09.264670 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:09.270405 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-fc436b75-0302-48d0-b65f-6c3f7207a724 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:09.279322 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['5085dbf1-e506-4928-8a5c-fa72496efeb5'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:04:09.316575 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Pushing event deleted for resources: {'Port': ['ID=5085dbf1-e506-4928-8a5c-fa72496efeb5,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:09.322285 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:09.407681 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:09.480543 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] No DVR routers for this DVR port 5085dbf1-e506-4928-8a5c-fa72496efeb5 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:04:09.483068 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:09.485728 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '55072c0c-2294-4520-886d-0beec11f9b50', 'name': 'network-vif-deleted', 'tag': '5085dbf1-e506-4928-8a5c-fa72496efeb5'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:04:09.487944 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-04aa5153-c09c-456b-873c-cd6660931ef2" -d '{"events": [{"server_uuid": "55072c0c-2294-4520-886d-0beec11f9b50", "name": "network-vif-deleted", "tag": "5085dbf1-e506-4928-8a5c-fa72496efeb5"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:04:09.533247 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:04:09 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-fbb66fec-4daa-4540-93a2-f1e353aabfb6 x-openstack-request-id: req-fbb66fec-4daa-4540-93a2-f1e353aabfb6 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:04:09.533598 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "55072c0c-2294-4520-886d-0beec11f9b50", "name": "network-vif-deleted", "tag": "5085dbf1-e506-4928-8a5c-fa72496efeb5", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:04:09.533883 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-fbb66fec-4daa-4540-93a2-f1e353aabfb6 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:04:09.534308 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '55072c0c-2294-4520-886d-0beec11f9b50', 'name': 'network-vif-deleted', 'tag': '5085dbf1-e506-4928-8a5c-fa72496efeb5', 'status': 'completed', 'code': 200} May 31 12:04:09.566778 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cb9e5a25-ab8b-4b45-8255-4cc5404727a3 req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.47 "DELETE /v2.0/ports/5085dbf1-e506-4928-8a5c-fa72496efeb5 HTTP/1.1" status: 204 len: 173 time: 0.8674023 May 31 12:04:09.683795 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b160327c-7ef3-4f6d-bfd2-2ecff0b00d8e req-e044311e-98cd-4cd4-a9e7-1e91244e7251 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 210 time: 0.1335895 May 31 12:04:10.838556 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-a69736af-271f-4a76-bd7d-e609c78d28c7 req-dec2e75a-2112-489d-85cc-000f36602d1a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Synchronizing usage tracker for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:04:10.920054 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-a69736af-271f-4a76-bd7d-e609c78d28c7 req-dec2e75a-2112-489d-85cc-000f36602d1a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Unset dirty status for tenant:e0fbc2aa94f84d079cbe91bba91e2516 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:04:10.921173 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a69736af-271f-4a76-bd7d-e609c78d28c7 req-dec2e75a-2112-489d-85cc-000f36602d1a tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 210 time: 0.1246111 May 31 12:04:12.087672 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-880e97ed-9581-478f-bbff-4ce289bd4c0c req-da2510d7-bade-4788-91a7-0f4775eeb124 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] 10.222.0.48 "GET /v2.0/ports?device_id=55072c0c-2294-4520-886d-0beec11f9b50 HTTP/1.1" status: 200 len: 210 time: 0.0462241 May 31 12:04:12.418842 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5085dbf1-e506-4928-8a5c-fa72496efeb5 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:04:12.455198 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 5085dbf1-e506-4928-8a5c-fa72496efeb5 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:04:12.456351 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5085dbf1-e506-4928-8a5c-fa72496efeb5 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:04:20.903205 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f8d07bdc-3de6-4ce5-b956-9dcc516e1076 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:23.337863 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:23.485587 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8dec6275-98b4-48b2-9c9c-e7411900cdde None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:24.612255 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:26.121536 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:31.463672 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:04:33.236528 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40014) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:33.298689 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Deleting subnet 1d9b77a2-8a70-4491-974c-8c516e56cff2 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:04:33.301632 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:33.352350 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:33.405887 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:33.721395 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:33.724952 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:33.764462 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Pushing event deleted for resources: {'Subnet': ['ID=1d9b77a2-8a70-4491-974c-8c516e56cff2,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:33.769346 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:33.777914 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.778285 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.778647 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.779000 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.779348 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.779698 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.780041 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.780382 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.780748 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.781051 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.781402 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.786170 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:33.786170 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] 10.222.0.48 "DELETE /v2.0/subnets/1d9b77a2-8a70-4491-974c-8c516e56cff2 HTTP/1.1" status: 204 len: 168 time: 0.5479190 May 31 12:04:33.786366 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-957f2d1d-2098-4fa2-be19-642438cca7be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:33.787987 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40022) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:33.898716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.021938 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.035868 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['1b0e8409-da86-4bc0-b712-3072314a6793']}} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:34.048854 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90') {'for_net_delete': True} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:34.052442 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '696a6e36-d4a5-49a4-8c95-8fe4a1d6aa90') {} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:34.060819 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3096822'] for segment, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.077711 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.083618 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Releasing vlan 354 on physical network physnet1 to pool {{(pid=99040) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:04:34.083996 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2278144'] for segment, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: Traceback (most recent call last): May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: timer() May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: cb(*args, **kw) May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: return func(*args, **kwargs) May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: self._notify() May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: self.callback(batched_events) May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: event.method(event) May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: return f(self, *a, **k) May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._get(url).json() May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: return self.request(url, 'GET', **kwargs) May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:04:34.097772 dv-cinder-791239-4-iscsi neutron-server[99040]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:04:34.246939 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.250551 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-374fd340-7868-41db-af9c-5c91f381e781 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:34.263849 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.273906 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Pushing event deleted for resources: {'Network': ['ID=1b0e8409-da86-4bc0-b712-3072314a6793,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:34.282973 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.283342 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.283700 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.284072 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.284399 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.284744 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.285088 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.285429 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.285802 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.286144 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.286483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.286818 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:34.290275 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:34.373407 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-5cda6a61-64f4-4aa7-bb0d-f1cbb8374ccb admin admin] 10.222.0.48 "DELETE /v2.0/networks/1b0e8409-da86-4bc0-b712-3072314a6793 HTTP/1.1" status: 204 len: 168 time: 0.5843568 May 31 12:04:34.680788 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40032) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:34.728424 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-2aadb569-2b60-443f-ab82-e94443a492ec admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=e946adaaf3ee413da7e5acaa60376c5b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0464807 May 31 12:04:34.730721 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40036) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:34.792455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9a7fd613-cd64-430b-a3e6-cf7e96eac84f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.817840 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9a7fd613-cd64-430b-a3e6-cf7e96eac84f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.966791 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9a7fd613-cd64-430b-a3e6-cf7e96eac84f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:34.970479 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-355fd2e9-da1e-41d4-9a37-feb8d3024d26 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:34.983977 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a7fd613-cd64-430b-a3e6-cf7e96eac84f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2e80e113-7a5f-431b-b1bc-fdc03edd754f,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:34.986535 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9a7fd613-cd64-430b-a3e6-cf7e96eac84f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:35.075271 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-9a7fd613-cd64-430b-a3e6-cf7e96eac84f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:35.076324 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-9a7fd613-cd64-430b-a3e6-cf7e96eac84f admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/2e80e113-7a5f-431b-b1bc-fdc03edd754f HTTP/1.1" status: 204 len: 168 time: 0.3447392 May 31 12:04:39.394339 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:41.128611 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40170) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:41.426223 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Deleting port babc9027-ff78-4d33-b481-f3e9ce412dfc {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:41.426562 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:41.562273 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:41.562589 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:41.722850 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Calling delete_port for babc9027-ff78-4d33-b481-f3e9ce412dfc owned by network:floatingip {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:42.001486 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:42.005174 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:42.076008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event deleted for resources: {'Port': ['ID=babc9027-ff78-4d33-b481-f3e9ce412dfc,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:42.082703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:42.145812 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for floatingip, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:42.151439 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Floating IP d49963a0-0633-44de-951e-20f07f82e459 disassociated (deleted). External IP: 172.27.1.240, port: 5add15c1-9f55-440c-b343-5d2f806a63c2. May 31 12:04:42.188517 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40178) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:42.252706 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Deleting subnet e4a37872-4de9-422e-a54c-71186b6f99d7 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:04:42.256945 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:42.328002 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:42.388088 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:42.569813 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-4b3b4ff2-c820-4f56-99ba-544803734ac3 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1813807 May 31 12:04:42.727102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Router 0da52ba7-9067-4610-8053-ced8a908f89b has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:04:42.762448 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:42.767107 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:42.802090 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-a136f511-5762-4016-9811-ba3aa094ed75 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:42.803753 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-a136f511-5762-4016-9811-ba3aa094ed75 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1098 time: 0.1549230 May 31 12:04:42.811795 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Pushing event deleted for resources: {'Subnet': ['ID=e4a37872-4de9-422e-a54c-71186b6f99d7,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:42.815688 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:42.818994 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:42.827045 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] 10.222.0.48 "DELETE /v2.0/subnets/e4a37872-4de9-422e-a54c-71186b6f99d7 HTTP/1.1" status: 204 len: 168 time: 0.6374686 May 31 12:04:42.829483 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40184) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:42.942360 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:43.073717 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:04:43.073950 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:43.084884 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-413ba3e5-b7f7-4b24-8f6c-403aa9723836 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:43.086459 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-413ba3e5-b7f7-4b24-8f6c-403aa9723836 service neutron] 10.222.0.47 "GET /v2.0/networks?id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 200 len: 910 time: 0.2595193 May 31 12:04:43.087745 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['ad7523f0-ba37-4688-b7be-179e0e8eae53']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:43.090041 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] Sync routers for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:04:43.101417 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '865e26f1-d91f-476e-a356-62b51f1cbd7b') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:43.102133 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '865e26f1-d91f-476e-a356-62b51f1cbd7b') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:43.103392 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:43.104670 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:43.114220 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:43.130637 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:43.143251 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Releasing vlan 382 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:04:43.143736 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:43.157688 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-4742e0b4-93d7-415e-8d0b-99b56a3174dd service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 200 len: 217 time: 0.0633814 May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:04:43.158284 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:04:43.164277 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:43.272936 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "DELETE /v2.0/floatingips/d49963a0-0633-44de-951e-20f07f82e459 HTTP/1.1" status: 204 len: 168 time: 2.1432371 May 31 12:04:43.273812 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:04:43.278403 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7" -d '{"events": [{"name": "network-changed", "server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:04:43.281986 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-639145da-a16f-4917-a11e-0a35b121716a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:43.283325 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-639145da-a16f-4917-a11e-0a35b121716a service neutron] 10.222.0.47 "GET /v2.0/subnets?id=3a19e015-3084-4732-bb65-ea124df3118f HTTP/1.1" status: 200 len: 848 time: 0.0873017 May 31 12:04:43.297501 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:43.316057 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:43.322886 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:43.352136 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:43.368008 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Pushing event deleted for resources: {'Network': ['ID=ad7523f0-ba37-4688-b7be-179e0e8eae53,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:43.375733 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:43.376134 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:04:43 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-324afa31-ba47-4598-a4c3-293d1a06fdb2 x-openstack-request-id: req-324afa31-ba47-4598-a4c3-293d1a06fdb2 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:04:43.376463 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:04:43.376751 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-324afa31-ba47-4598-a4c3-293d1a06fdb2 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:04:43.377174 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'status': 'completed', 'code': 200} May 31 12:04:43.414194 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-ef8fe273-989e-4713-946f-0f7574b73223 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:43.415685 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-ef8fe273-989e-4713-946f-0f7574b73223 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1254187 May 31 12:04:43.493801 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1a5781b3-1f04-4a7f-aaf3-20217e7776f7 admin admin] 10.222.0.48 "DELETE /v2.0/networks/ad7523f0-ba37-4688-b7be-179e0e8eae53 HTTP/1.1" status: 204 len: 168 time: 0.6632473 May 31 12:04:43.550010 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-f5bfddf0-d311-4512-8d11-69fd12a773f9 service neutron] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99040) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 12:04:43.551172 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-f5bfddf0-d311-4512-8d11-69fd12a773f9 service neutron] 10.222.0.47 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.1275468 May 31 12:04:43.686923 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:57Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:43.691479 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] Checking router: 0da52ba7-9067-4610-8053-ced8a908f89b for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:04:43.700415 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:43.701050 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:43.707900 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-7fa9f0c7-b6c0-4f09-b7bc-8b87f8ef81da service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1494005 May 31 12:04:43.855611 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1dff5b39-2bdd-4a4b-b17a-5afbf6fe3117 req-c48774e1-1654-4a94-94fb-39bfd3cb9c44 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1406522 May 31 12:04:43.903495 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:43.933511 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40204) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:44.015065 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-aced1f21-d6e6-4cb3-b02d-0e668a462528 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=e0fbc2aa94f84d079cbe91bba91e2516&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0802534 May 31 12:04:44.019427 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40208) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:44.086637 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-147fc0f7-92ba-48e5-9c73-a322c9d589dd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:44.121432 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-147fc0f7-92ba-48e5-9c73-a322c9d589dd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:44.143747 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-fe810de3-a462-4f48-a254-4239cea6d9b9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:44.145286 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-fe810de3-a462-4f48-a254-4239cea6d9b9 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=bbcb901221a14602a1f2f8b198287636&device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 1098 time: 0.1472979 May 31 12:04:44.285392 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-147fc0f7-92ba-48e5-9c73-a322c9d589dd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:44.292752 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd1672d1-f6cf-41f4-a75d-1b14adaa4cba req-e806a2f0-7fce-49f2-ad73-e3bc49a12e6a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:44.293553 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-cd1672d1-f6cf-41f4-a75d-1b14adaa4cba req-e806a2f0-7fce-49f2-ad73-e3bc49a12e6a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Synchronizing usage tracker for tenant:bbcb901221a14602a1f2f8b198287636 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:04:44.293706 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-11401e52-664d-4e95-87ed-4eb912fd60a8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:44.295407 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:57Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:44.301407 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c943d5df-2187-4bbf-9d4f-d3862e71a657 None None] The sync data for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi is: [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T11:59:51Z', 'revision_number': 4, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '824b42f8-92ee-4c2c-97e5-fdb7ab56aca3', 'name': '', 'network_id': 'fc7e08b1-448f-4bc6-8c22-cdf76e730c2f', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'mac_address': 'fa:16:3e:fa:17:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_interface', 'standard_attr_id': 510, 'fixed_ips': [{'subnet_id': '3a19e015-3084-4732-bb65-ea124df3118f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:50Z', 'updated_at': '2021-05-31T11:59:57Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'subnets': [{'id': '3a19e015-3084-4732-bb65-ea124df3118f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:04:44.424743 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-a9ee5830-c6db-4e0e-b479-0c802826d87d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:44.426206 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-a9ee5830-c6db-4e0e-b479-0c802826d87d service neutron] 10.222.0.47 "GET /v2.0/networks?id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 200 len: 910 time: 0.2719178 May 31 12:04:44.429607 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-147fc0f7-92ba-48e5-9c73-a322c9d589dd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=48250270-c82c-483c-a3cc-9d463d413e15,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:44.432004 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-147fc0f7-92ba-48e5-9c73-a322c9d589dd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:44.484760 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-cd1672d1-f6cf-41f4-a75d-1b14adaa4cba req-e806a2f0-7fce-49f2-ad73-e3bc49a12e6a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Unset dirty status for tenant:bbcb901221a14602a1f2f8b198287636 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:04:44.485967 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd1672d1-f6cf-41f4-a75d-1b14adaa4cba req-e806a2f0-7fce-49f2-ad73-e3bc49a12e6a tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.6087205 May 31 12:04:44.503964 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-8519e368-2565-47c4-a220-354ed30476a4 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 200 len: 217 time: 0.0700221 May 31 12:04:44.519957 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-147fc0f7-92ba-48e5-9c73-a322c9d589dd admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:44.523477 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-147fc0f7-92ba-48e5-9c73-a322c9d589dd admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/48250270-c82c-483c-a3cc-9d463d413e15 HTTP/1.1" status: 204 len: 168 time: 0.5006633 May 31 12:04:44.587120 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9ad72500-162d-40b8-8dfb-e7498f8a0622 None None] New status for floating IP d49963a0-0633-44de-951e-20f07f82e459: DOWN {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:04:44.604390 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cd1672d1-f6cf-41f4-a75d-1b14adaa4cba req-154aac96-09d4-46ed-8b6c-5bb752d1b181 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1030030 May 31 12:04:44.613711 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-9bd00682-9df5-496c-bf31-0287f2712576 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:44.615235 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-9bd00682-9df5-496c-bf31-0287f2712576 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=3a19e015-3084-4732-bb65-ea124df3118f HTTP/1.1" status: 200 len: 848 time: 0.0912113 May 31 12:04:44.711195 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-9ee5e2bc-cb0b-4f88-a38a-8d9676fe8077 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:44.712695 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-9ee5e2bc-cb0b-4f88-a38a-8d9676fe8077 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0846627 May 31 12:04:44.850206 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-55123a26-91b5-4fc8-b844-9d540f4e8ea0 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1296754 May 31 12:04:44.996515 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b7630e1c-a4ef-4d61-bc73-41315e04d7f7 req-becba3b2-18c9-448c-807f-e59626157e58 service neutron] 10.222.0.47 "GET /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1327734 May 31 12:04:45.493520 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-8a884c97-995a-4859-b73a-46b9a3f80aa7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:45.495906 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-8a884c97-995a-4859-b73a-46b9a3f80aa7 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.47 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.0871620 May 31 12:04:45.641793 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Deleting port 5add15c1-9f55-440c-b343-5d2f806a63c2 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:45.642638 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:45.734301 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40236) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:45.788687 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] For port 5add15c1-9f55-440c-b343-5d2f806a63c2, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5add15c1-9f55-440c-b343-5d2f806a63c2,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:45.789034 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:45.916121 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-340341230-network', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:04:45.917370 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:04:45.925855 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Calling delete_port for 5add15c1-9f55-440c-b343-5d2f806a63c2 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:45.931085 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Usage tracker for resource:network and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:04:45.935627 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:04:45.942650 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:04:46.009186 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a3d29874-827d-48be-a464-d9d4a72c7319 req-4f07ba7f-d623-4729-a74d-43a26f4a6901 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:46.012179 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a3d29874-827d-48be-a464-d9d4a72c7319 req-4f07ba7f-d623-4729-a74d-43a26f4a6901 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 945 time: 0.2799442 May 31 12:04:46.031892 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Made reservation on behalf of 0e2c6e0e5c334c60a47f2dc2d0c79653 for: {'network': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:04:46.034448 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.059236 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.060395 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a3d29874-827d-48be-a464-d9d4a72c7319 req-7a36f21a-9b99-4d5e-adba-b96c4e58651b tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/security-groups?id=3bc79f51-0858-405c-97be-c79308faa377&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0421417 May 31 12:04:46.081770 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:04:46.090854 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Usage tracker for resource:security_group_rule and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:04:46.094838 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:04:46.100923 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:04:46.195497 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.202801 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6e6a091e-17a0-4a7b-bacf-de13587ce1f9 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:46.256233 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event deleted for resources: {'Port': ['ID=5add15c1-9f55-440c-b343-5d2f806a63c2,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:46.330079 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:46.342194 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] No DVR routers for this DVR port 5add15c1-9f55-440c-b343-5d2f806a63c2 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:04:46.345099 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.347537 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'name': 'network-vif-deleted', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:04:46.349792 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-9e0d05a0-8c03-427f-8fcc-bce4c15618aa" -d '{"events": [{"server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "name": "network-vif-deleted", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:04:46.396157 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:04:46 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-74cd8d70-583a-46e6-ad88-78524fde6c5f x-openstack-request-id: req-74cd8d70-583a-46e6-ad88-78524fde6c5f {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:04:46.396481 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "e1a677ae-cce8-4a8c-83e3-878361e41314", "name": "network-vif-deleted", "tag": "5add15c1-9f55-440c-b343-5d2f806a63c2", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:04:46.396764 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-74cd8d70-583a-46e6-ad88-78524fde6c5f {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:04:46.397178 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e1a677ae-cce8-4a8c-83e3-878361e41314', 'name': 'network-vif-deleted', 'tag': '5add15c1-9f55-440c-b343-5d2f806a63c2', 'status': 'completed', 'code': 200} May 31 12:04:46.417579 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.481824 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5fa3c037-baf0-4cd5-8b90-9898cb87076d req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.47 "DELETE /v2.0/ports/5add15c1-9f55-440c-b343-5d2f806a63c2 HTTP/1.1" status: 204 len: 173 time: 0.9796326 May 31 12:04:46.538006 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.541326 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-df66d9b7-00ec-4786-88a2-215c99d11638 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:46.575125 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] vlan segment allocate from pool success with {'vlan_id': 362, 'physical_network': 'physnet1'} {{(pid=99040) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 12:04:46.580309 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7174e6c9-cf73-4a54-b506-efa9bb4198e0,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:46.583971 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:46.588053 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.588949 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.segments_db [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Added segment a29d566a-c32d-4472-972d-990571fdae77 of type vlan for network 8a4efbb8-aa93-49be-a073-41ce66e25d2f May 31 12:04:46.616881 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.737181 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.738558 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:46.739135 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:46.742738 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:46.924595 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Pushing event updated for resources: {'Network': ['ID=8a4efbb8-aa93-49be-a073-41ce66e25d2f,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:46.927218 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.184s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:46.934261 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 12:04:47.030629 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-2f866f32-36fd-4686-acd6-8e139dbe5174 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 1.2950528 May 31 12:04:47.033140 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40248) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:47.039004 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Request body: {'subnet': {'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-340341230-subnet', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'ip_version': 4}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:04:47.044364 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:04:47.048675 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Usage tracker for resource:subnet and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:04:47.051601 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:04:47.059494 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:04:47.217096 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-916f9566-ce7d-46da-b394-f830eb24e023 req-30bb87ef-d935-4946-b05b-1a3409e81e17 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "GET /v2.0/ports?device_id=e1a677ae-cce8-4a8c-83e3-878361e41314 HTTP/1.1" status: 200 len: 210 time: 0.0472860 May 31 12:04:47.228105 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Made reservation on behalf of 0e2c6e0e5c334c60a47f2dc2d0c79653 for: {'subnet': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:04:47.241262 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2831116'] for subnet, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:47.683133 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2111717', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2831130'] for subnet, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:47.687790 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a2844483-3dd9-467f-b976-be80e75d7f6e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:47.759106 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Pushing event updated for resources: {'Subnet': ['ID=4a96f417-9ce3-448a-9f7f-529c69a0ace7,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:47.762792 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:47.768764 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Checking segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:04:47.769115 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Network hostable DHCP agents. Network: 8a4efbb8-aa93-49be-a073-41ce66e25d2f, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99040) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 12:04:47.840218 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99040) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 12:04:47.921111 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 8a4efbb8-aa93-49be-a073-41ce66e25d2f) {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:04:47.933965 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-dddf66f3-4238-472c-ba4f-e2a8ca796b19 None None] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:04:47.939983 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:47.940575 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.030434 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 0.9958501 May 31 12:04:48.032845 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40256) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:48.038033 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:04:48.042687 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:04:48.046521 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Usage tracker for resource:router and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:04:48.052456 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:04:48.055227 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:04:48.127080 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.128294 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.145064 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.167100 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 12:04:48.167978 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.318122 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40264) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:48.341357 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Allocated IP 172.20.0.2 (8a4efbb8-aa93-49be-a073-41ce66e25d2f/4a96f417-9ce3-448a-9f7f-529c69a0ace7/b19fb3d5-4dc8-4350-871f-7964e82205d8) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:04:48.402916 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-270c33b9-9be5-439d-b66d-10b19658ad64 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.446635 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-270c33b9-9be5-439d-b66d-10b19658ad64 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.519443 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.580251 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5add15c1-9f55-440c-b343-5d2f806a63c2 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:04:48.601061 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-270c33b9-9be5-439d-b66d-10b19658ad64 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.606185 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-b448cb4f-3363-49e1-8fc8-e5206b442402 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:48.622064 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-270c33b9-9be5-439d-b66d-10b19658ad64 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=3bc79f51-0858-405c-97be-c79308faa377,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:48.624681 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-270c33b9-9be5-439d-b66d-10b19658ad64 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:48.651295 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 5add15c1-9f55-440c-b343-5d2f806a63c2 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:04:48.652394 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 5add15c1-9f55-440c-b343-5d2f806a63c2 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:04:48.657473 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.668313 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Transition to ACTIVE for port object b19fb3d5-4dc8-4350-871f-7964e82205d8 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:04:48.685735 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Transition to ACTIVE for port object b19fb3d5-4dc8-4350-871f-7964e82205d8 will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:04:48.708963 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-270c33b9-9be5-439d-b66d-10b19658ad64 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.715126 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-270c33b9-9be5-439d-b66d-10b19658ad64 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] 10.222.0.48 "DELETE /v2.0/security-groups/3bc79f51-0858-405c-97be-c79308faa377 HTTP/1.1" status: 204 len: 168 time: 0.3909850 May 31 12:04:48.784390 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.788194 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-8cca4ec6-b743-4bbb-a115-6ecd5688e4f3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:48.829068 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40270) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:48.863607 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:04:48.866506 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:04:48.869986 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:04:48.875744 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Pushing event updated for resources: {'Port': ['ID=b19fb3d5-4dc8-4350-871f-7964e82205d8,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:48.914830 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.914830 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.915216 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.915317 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.915559 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.915875 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.916779 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.917341 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.917777 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.918099 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.918406 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.918729 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:04:48.927763 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.949802 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:48.960112 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.980108 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Attempting to bind port b19fb3d5-4dc8-4350-871f-7964e82205d8 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:04:48.980858 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Attempting to bind port b19fb3d5-4dc8-4350-871f-7964e82205d8 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:04:48.981460 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Attempting to bind port b19fb3d5-4dc8-4350-871f-7964e82205d8 on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:04:48.987839 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 4, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:04:48.988583 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Checking segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:04:48.989224 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:48.990080 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Bound using segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:04:48.990989 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Bound port: b19fb3d5-4dc8-4350-871f-7964e82205d8, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:04:48.991487 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:49.100066 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] For port b19fb3d5-4dc8-4350-871f-7964e82205d8, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:04:49.104661 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] For port b19fb3d5-4dc8-4350-871f-7964e82205d8, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b19fb3d5-4dc8-4350-871f-7964e82205d8,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:04:49.106737 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Allocated IP 172.27.1.66 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/3410a2bc-9d4c-4de9-b287-f0317f808672) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:04:49.184849 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:49.189029 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:49.288409 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Pushing event updated for resources: {'Port': ['ID=b19fb3d5-4dc8-4350-871f-7964e82205d8,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:49.293253 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-692bb2fd-dd59-4cdf-8308-83b6221da80b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:49.371952 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:49.526032 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:49.539736 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:49.539736 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:49.622725 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:04:49.628288 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:04:49.632371 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:04:49.666975 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Pushing event updated for resources: {'Port': ['ID=3410a2bc-9d4c-4de9-b287-f0317f808672,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:49.673142 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:49.963541 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Deleting port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:49.963879 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:50.093341 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] For port 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:50.093726 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:50.124138 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:50.294769 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-7e32d289-416d-479d-913f-2c8c7b792979 None None] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:04:50.328449 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Calling delete_port for 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 owned by network:router_interface {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:50.454298 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-61d10f03-3ade-4bb5-95e9-ea8fd13c9945 None None] Provisioning for port b19fb3d5-4dc8-4350-871f-7964e82205d8 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:04:50.497964 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:50.566226 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:50.578393 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-bc808800-1a53-4df1-8cbd-51707d25b915 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:50.626684 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Pushing event deleted for resources: {'Port': ['ID=824b42f8-92ee-4c2c-97e5-fdb7ab56aca3,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:50.710981 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:50.812644 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99040) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 12:04:50.915275 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9c5ecd27-4ab8-4325-832b-a6fdaa0b6d0d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:50.929770 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:50.995536 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b19fb3d5-4dc8-4350-871f-7964e82205d8 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:04:51.023437 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:04:51.036492 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:04:51.058083 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:51.058810 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:51.132682 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Notify callbacks [] for router, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:51.134237 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 922 time: 3.1002870 May 31 12:04:51.137585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40292) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:51.249907 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:51.308890 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Router 0da52ba7-9067-4610-8053-ced8a908f89b has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:04:51.356088 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port b19fb3d5-4dc8-4350-871f-7964e82205d8 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:04:51.375167 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port b19fb3d5-4dc8-4350-871f-7964e82205d8 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:04:51.376417 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:51.505605 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:04:51.522154 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-726e3e0b-356f-4c7f-b4c7-263b0397aba8 admin admin] 10.222.0.48 "PUT /v2.0/routers/0da52ba7-9067-4610-8053-ced8a908f89b/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.6895187 May 31 12:04:51.522368 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] Sync routers for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:04:51.522838 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40296) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:51.530639 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:51.531245 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:51.615750 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:51.659761 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port b19fb3d5-4dc8-4350-871f-7964e82205d8, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b19fb3d5-4dc8-4350-871f-7964e82205d8,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:51.674178 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:49Z', 'revision_number': 3, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:49Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:51.678358 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:04:51.678716 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Updating router 5b6f052e-0b0b-4603-9141-bb2a360accdf port 3410a2bc-9d4c-4de9-b287-f0317f808672 binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 12:04:51.722321 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:51.743833 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:51.778774 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:51.786768 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks [] for router, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:51.789884 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:51.940634 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=b19fb3d5-4dc8-4350-871f-7964e82205d8,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:51.944963 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:52.020933 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.068756 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T12:04:50Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:52.072679 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] Checking router: 0da52ba7-9067-4610-8053-ced8a908f89b for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:04:52.086806 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] Routers {'0da52ba7-9067-4610-8053-ced8a908f89b'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:52.087691 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0da52ba7-9067-4610-8053-ced8a908f89b'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:52.149519 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.261764 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.262481 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Allocated IP 172.20.0.1 (8a4efbb8-aa93-49be-a073-41ce66e25d2f/4a96f417-9ce3-448a-9f7f-529c69a0ace7/077741d4-ebe8-4cb3-9d13-97cd5c09d04c) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:04:52.278174 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Transition to ACTIVE for port object 3410a2bc-9d4c-4de9-b287-f0317f808672 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:04:52.296122 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] Set router 0da52ba7-9067-4610-8053-ced8a908f89b gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:52.302325 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Deleting port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:52.302667 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.366579 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.370932 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-c69390d6-5260-479c-8eda-b9f4b35af6a5 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:52.429647 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] For port a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:52.430002 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.446351 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.448361 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Attempting to bind port 3410a2bc-9d4c-4de9-b287-f0317f808672 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:04:52.448854 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Attempting to bind port 3410a2bc-9d4c-4de9-b287-f0317f808672 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:04:52.449229 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Attempting to bind port 3410a2bc-9d4c-4de9-b287-f0317f808672 on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:04:52.452718 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ad1e9d7a-4053-4df1-ad48-47108932cf7b None None] Pushing event updated for resources: {'Port': ['ID=3410a2bc-9d4c-4de9-b287-f0317f808672,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:52.456867 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-ad1e9d7a-4053-4df1-ad48-47108932cf7b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:52.457533 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 4, 23), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:04:52.457956 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:04:52.458367 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.458758 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:04:52.459238 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Bound port: 3410a2bc-9d4c-4de9-b287-f0317f808672, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:04:52.459618 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.467160 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Transition to ACTIVE for port object 077741d4-ebe8-4cb3-9d13-97cd5c09d04c will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:04:52.541963 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] For port 3410a2bc-9d4c-4de9-b287-f0317f808672, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:04:52.545046 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] For port 3410a2bc-9d4c-4de9-b287-f0317f808672, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=3410a2bc-9d4c-4de9-b287-f0317f808672,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:04:52.561187 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.565013 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f86d4818-dd7e-4ae0-853c-e2543825f2fa tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:52.569509 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Calling delete_port for a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 owned by network:router_gateway {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:52.612989 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T12:04:50Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:52.617714 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-35435f42-15dd-4aa4-b49d-8ae65b5aa061 None None] The sync data for ids ['0da52ba7-9067-4610-8053-ced8a908f89b'] in dv-cinder-791239-4-iscsi is: [{'id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'name': 'tempest-TestShelveInstance-1197050931-router', 'tenant_id': 'bbcb901221a14602a1f2f8b198287636', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158'}], 'enable_snat': True}, 'gw_port_id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:45Z', 'updated_at': '2021-05-31T12:04:50Z', 'revision_number': 5, 'project_id': 'bbcb901221a14602a1f2f8b198287636', 'gw_port': {'id': 'a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:71:e2:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0da52ba7-9067-4610-8053-ced8a908f89b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 509, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:47Z', 'updated_at': '2021-05-31T11:59:53Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:04:52.629388 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:04:52.632369 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:04:52.636773 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:04:52.641059 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.650248 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:52.712271 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Pushing event updated for resources: {'Port': ['ID=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:52.717002 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.152s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:52.719788 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.737376 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:52.738266 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:52.818125 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:52.826397 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-4311a1e5-36f5-46fd-b35d-68a5f6d68c04 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:52.829306 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ad1e9d7a-4053-4df1-ad48-47108932cf7b None None] Pushing event updated for resources: {'Port': ['ID=3410a2bc-9d4c-4de9-b287-f0317f808672,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:52.832453 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-ad1e9d7a-4053-4df1-ad48-47108932cf7b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.182s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:52.881842 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Pushing event deleted for resources: {'Port': ['ID=a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:52.884314 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:52.952468 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:53.016417 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.068161 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-2a28ff27-61d0-4d0f-95f9-644e078bb1b5 None None] Provisioning for port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:04:53.173241 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b19fb3d5-4dc8-4350-871f-7964e82205d8 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:04:53.201475 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-2a28ff27-61d0-4d0f-95f9-644e078bb1b5 None None] Provisioning complete for port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:04:53.202014 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2a28ff27-61d0-4d0f-95f9-644e078bb1b5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.223987 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.226324 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.242901 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.315225 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.316186 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Fanout notify agent at l3_agent the message router_deleted on router 0da52ba7-9067-4610-8053-ced8a908f89b {{(pid=99039) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 12:04:53.329151 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.330503 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-f9e0e81c-ad14-488e-a91f-de6b12cf751a admin admin] 10.222.0.48 "DELETE /v2.0/routers/0da52ba7-9067-4610-8053-ced8a908f89b HTTP/1.1" status: 204 len: 168 time: 1.8033905 May 31 12:04:53.332913 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40304) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:53.334942 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-2a28ff27-61d0-4d0f-95f9-644e078bb1b5 None None] Port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:04:53.409223 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] Deleting subnet 3a19e015-3084-4732-bb65-ea124df3118f {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:04:53.414599 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.415127 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.426250 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port b19fb3d5-4dc8-4350-871f-7964e82205d8 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:04:53.426627 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.473508 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.512763 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.513663 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:49Z', 'revision_number': 3, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:53.517621 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6bd51b5d-0c7c-415b-91e6-7117ec1a9c92 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.547285 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6aa54725-6030-4057-8458-df4bbfa9377f None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:49Z', 'revision_number': 3, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:04:53.561334 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.578170 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.673861 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:04:53.674411 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-410ff5ac-3482-476a-b395-68fb26b94588 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:53.674510 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:04:53.674510 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 824b42f8-92ee-4c2c-97e5-fdb7ab56aca3 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:04:53.689311 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.693655 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Pushing event updated for resources: {'Port': ['ID=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:53.698468 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:53.826826 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:04:53.827961 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:04:53.909954 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.914800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:53.958751 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=3a19e015-3084-4732-bb65-ea124df3118f,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:53.969888 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:53.972076 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-fce45d1e-2734-4604-81e5-a0691a5c2a62 None None] Network fc7e08b1-448f-4bc6-8c22-cdf76e730c2f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:04:53.972458 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:53.976630 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-8a90c70f-3939-4ac8-9ad4-442bf5940b52 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/3a19e015-3084-4732-bb65-ea124df3118f HTTP/1.1" status: 204 len: 168 time: 0.6427388 May 31 12:04:53.979049 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40308) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:54.136607 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:04:54.264800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:54.376628 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Deleting port 28d2e39a-c647-40fa-a4f8-8a846693f680 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:54.376628 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:54.417217 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:04:54.419878 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-afe995d6-7259-4cc2-a7d3-cd6643041260 None None] Provisioning complete for port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:04:54.420405 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-afe995d6-7259-4cc2-a7d3-cd6643041260 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:54.426982 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] 10.222.0.48 "PUT /v2.0/routers/5b6f052e-0b0b-4603-9141-bb2a360accdf/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.2883880 May 31 12:04:54.521984 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-afe995d6-7259-4cc2-a7d3-cd6643041260 None None] Port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:04:54.554880 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] For port 28d2e39a-c647-40fa-a4f8-8a846693f680, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=28d2e39a-c647-40fa-a4f8-8a846693f680,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:54.555244 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:54.605308 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] DHCP port deletion for fc7e08b1-448f-4bc6-8c22-cdf76e730c2f request from dv-cinder-791239-4-iscsi {{(pid=99041) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 31 12:04:54.651313 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:54.699228 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] Deleting port 28d2e39a-c647-40fa-a4f8-8a846693f680 {{(pid=99041) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:54.699818 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:54.718800 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Calling delete_port for 28d2e39a-c647-40fa-a4f8-8a846693f680 owned by network:dhcp {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:54.810684 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] For port 28d2e39a-c647-40fa-a4f8-8a846693f680, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=28d2e39a-c647-40fa-a4f8-8a846693f680,segment=NetworkSegment(edd9824c-4e69-4a31-bd8c-e501618fb7bb),segment_id=edd9824c-4e69-4a31-bd8c-e501618fb7bb)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:54.811241 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:54.834978 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:54.841279 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:54.887718 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Pushing event deleted for resources: {'Port': ['ID=28d2e39a-c647-40fa-a4f8-8a846693f680,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:54.891150 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:54.891922 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] No DVR routers for this DVR port 28d2e39a-c647-40fa-a4f8-8a846693f680 on host dv-cinder-791239-4-iscsi {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:04:54.953354 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] Calling delete_port for 28d2e39a-c647-40fa-a4f8-8a846693f680 owned by network:dhcp {{(pid=99041) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:54.967752 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 3410a2bc-9d4c-4de9-b287-f0317f808672 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:04:55.030950 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.046463 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['fc7e08b1-448f-4bc6-8c22-cdf76e730c2f']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:55.062754 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'edd9824c-4e69-4a31-bd8c-e501618fb7bb') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:55.063049 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'edd9824c-4e69-4a31-bd8c-e501618fb7bb') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:55.064865 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.069524 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-ad1e9d7a-4053-4df1-ad48-47108932cf7b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:55.075857 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.103347 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.126100 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Releasing vlan 397 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:04:55.126100 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.130288 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] Pushing event deleted for resources: {'Port': ['ID=28d2e39a-c647-40fa-a4f8-8a846693f680,revision_number=None']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:55.131872 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:55.134985 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-de431348-f6ca-40a6-95f9-30c36a147ad3 None None] No DVR routers for this DVR port 28d2e39a-c647-40fa-a4f8-8a846693f680 on host dv-cinder-791239-4-iscsi {{(pid=99041) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:04:55.137436 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:04:55.172284 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 3410a2bc-9d4c-4de9-b287-f0317f808672 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:04:55.178005 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 3410a2bc-9d4c-4de9-b287-f0317f808672 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:04:55.178558 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.315632 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.323006 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.323177 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-0c6798d9-88d1-4c2e-a029-ef04c5692aac admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:55.341437 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 3410a2bc-9d4c-4de9-b287-f0317f808672, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=3410a2bc-9d4c-4de9-b287-f0317f808672,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:55.347550 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.350122 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Pushing event deleted for resources: {'Network': ['ID=fc7e08b1-448f-4bc6-8c22-cdf76e730c2f,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:55.353064 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:55.435418 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.437596 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:55.451864 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1e6ab15c-6235-49e7-b181-ccc6fc4c13b9 admin admin] 10.222.0.48 "DELETE /v2.0/networks/fc7e08b1-448f-4bc6-8c22-cdf76e730c2f HTTP/1.1" status: 204 len: 168 time: 1.4687254 May 31 12:04:55.452234 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40334) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:55.496892 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40338) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:55.511625 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-750418318-network', 'tenant_id': '34d7d5411875496f938fb5fb58f45223'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:04:55.513063 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:04:55.513559 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:04:55.517061 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Usage tracker for resource:network and tenant:34d7d5411875496f938fb5fb58f45223 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:04:55.520007 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:04:55.526360 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:04:55.565761 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:04:55.566201 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device a3a2d113-ceb1-4a3f-95da-8a9b01e96ce4 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:04:55.566625 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 28d2e39a-c647-40fa-a4f8-8a846693f680 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:04:55.598909 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:04:55.611736 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 28d2e39a-c647-40fa-a4f8-8a846693f680 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:04:55.612090 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 28d2e39a-c647-40fa-a4f8-8a846693f680 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:04:55.612297 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Made reservation on behalf of 34d7d5411875496f938fb5fb58f45223 for: {'network': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:04:55.614814 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1458966'] for network, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.625039 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=3410a2bc-9d4c-4de9-b287-f0317f808672,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:55.627675 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:55.628416 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:55.634563 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.652662 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:04:55.656828 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Usage tracker for resource:security_group_rule and tenant:34d7d5411875496f938fb5fb58f45223 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:04:55.659870 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:04:55.666165 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:04:55.678362 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.241s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:55.758472 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.809048 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:55.829031 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.833178 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-06399369-3db8-443f-936e-2b624109c0db admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:55.861923 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] vlan segment allocate from pool success with {'vlan_id': 370, 'physical_network': 'physnet1'} {{(pid=99039) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 31 12:04:55.871636 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=524cc778-95f3-429b-802b-40668b0604e0,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:55.874849 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:55.878790 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:55.879128 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.db.segments_db [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Added segment 9bd3a04f-0739-4430-9c94-924934a7ab52 of type vlan for network 219aba32-474c-4847-b9d2-5a8ddc5c5369 May 31 12:04:55.907557 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8759697012053'] for network, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.023824 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023'] for network, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.026522 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:56.026522 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.032022 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-47e85dfa-1a94-4f97-a464-6be2a3207310 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:56.133658 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Pushing event updated for resources: {'Network': ['ID=219aba32-474c-4847-b9d2-5a8ddc5c5369,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:56.135463 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.136782 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:56.144798 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Network 219aba32-474c-4847-b9d2-5a8ddc5c5369 details requested by agent ovs-agent-dv-cinder-791239-4-iscsi with host dv-cinder-791239-4-iscsi {{(pid=99041) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 31 12:04:56.247192 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-a388b339-0be4-4ba7-872c-69583c33a669 admin admin] 10.222.0.48 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 0.7474449 May 31 12:04:56.250116 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40344) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:56.255543 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Request body: {'subnet': {'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-750418318-subnet', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'ip_version': 4}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:04:56.261381 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:04:56.267404 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Usage tracker for resource:subnet and tenant:34d7d5411875496f938fb5fb58f45223 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:04:56.267533 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.269014 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:04:56.273119 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:53Z', 'revision_number': 2, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:56.279114 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:04:56.287924 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:04:56.288287 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Updating router 5b6f052e-0b0b-4603-9141-bb2a360accdf port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 12:04:56.351017 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.377993 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Made reservation on behalf of 34d7d5411875496f938fb5fb58f45223 for: {'subnet': 1} {{(pid=99039) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:04:56.380423 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2330284'] for subnet, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.587151 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Deleting port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:56.588291 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.710330 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] For port e3e2dad9-58b2-48c0-8b94-ffde87a6b32a, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,segment=NetworkSegment(29648ab6-8c7b-47d7-9f64-2b32a50ee3f7),segment_id=29648ab6-8c7b-47d7-9f64-2b32a50ee3f7)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:56.710809 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.715252 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.727896 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Transition to ACTIVE for port object 077741d4-ebe8-4cb3-9d13-97cd5c09d04c will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:04:56.829743 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.838413 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Calling delete_port for e3e2dad9-58b2-48c0-8b94-ffde87a6b32a owned by network:router_interface {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:56.845742 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:56.893079 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2338130', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2330298'] for subnet, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:56.898131 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:56.951893 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 3410a2bc-9d4c-4de9-b287-f0317f808672 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:04:56.981353 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Pushing event updated for resources: {'Subnet': ['ID=03af805c-fc50-465f-bbbf-187db4b13f93,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:56.985891 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:56.986452 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Checking segment: {'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99039) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:04:56.986899 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Network hostable DHCP agents. Network: 219aba32-474c-4847-b9d2-5a8ddc5c5369, hostable agents: ['5f4d2945-70cd-4b26-a4ad-a4535f5a727e'], hosted agents: [] {{(pid=99039) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 31 12:04:57.063768 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Network 219aba32-474c-4847-b9d2-5a8ddc5c5369 is scheduled to be hosted by DHCP agent 5f4d2945-70cd-4b26-a4ad-a4535f5a727e with binding_index 1 {{(pid=99039) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 31 12:04:57.073878 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-612f4d10-3103-4f35-954c-9961bc5a91e3 None None] Pushing event updated for resources: {'Port': ['ID=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:57.109732 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-612f4d10-3103-4f35-954c-9961bc5a91e3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.264s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:57.142442 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Attempting to bind port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:04:57.142816 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Attempting to bind port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:04:57.143174 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Attempting to bind port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:04:57.156748 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 3410a2bc-9d4c-4de9-b287-f0317f808672 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:04:57.157121 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.172101 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-4-iscsi, 219aba32-474c-4847-b9d2-5a8ddc5c5369) {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:04:57.181108 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:04:57.182303 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.184937 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 4, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:04:57.185317 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Checking segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:04:57.185748 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.186141 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Bound using segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:04:57.186627 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Bound port: 077741d4-ebe8-4cb3-9d13-97cd5c09d04c, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:04:57.187006 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.202807 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-abd3d82d-ca68-46ee-8205-297376e2c556 None None] Network 219aba32-474c-4847-b9d2-5a8ddc5c5369 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:04:57.235044 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.240245 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:57.276068 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-fe258b69-d7ff-448a-93c7-b290f0c4ecbf admin admin] 10.222.0.48 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 1.0246005 May 31 12:04:57.278987 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40348) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:57.284275 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:04:57.289664 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:04:57.295075 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Usage tracker for resource:router and tenant:34d7d5411875496f938fb5fb58f45223 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:04:57.298686 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:04:57.305974 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:04:57.319334 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Pushing event deleted for resources: {'Port': ['ID=e3e2dad9-58b2-48c0-8b94-ffde87a6b32a,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:57.326233 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:57.423576 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-505677', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2073'] for router, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.424734 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-504653', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-507794', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-437631', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2066'] for router, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.431092 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] For port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:04:57.446706 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.480394 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] For port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:04:57.596439 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.601880 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:57.604937 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.635047 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp57fffe5e-a26d-5080-a1e8-32658770c67c-219aba32-474c-4847-b9d2-5a8ddc5c5369', 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'fixed_ips': [{'subnet_id': '03af805c-fc50-465f-bbbf-187db4b13f93'}]}} from dv-cinder-791239-4-iscsi. {{(pid=99041) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 31 12:04:57.636050 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:57.754329 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:57.755043 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:57.844707 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-612f4d10-3103-4f35-954c-9961bc5a91e3 None None] Pushing event updated for resources: {'Port': ['ID=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:57.848352 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-612f4d10-3103-4f35-954c-9961bc5a91e3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.247s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:57.995816 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-507831', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-505727', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2148'] for router, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:58.098809 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:58.136965 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Router e1489825-752a-4858-b711-8618e0a7bb8a has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:04:58.142831 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.db_base_plugin_common [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Allocated IP 172.20.0.2 (219aba32-474c-4847-b9d2-5a8ddc5c5369/03af805c-fc50-465f-bbbf-187db4b13f93/68599998-f433-42bf-80ae-c64a075f3712) {{(pid=99041) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:04:58.317310 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:04:58.330535 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] Sync routers for ids ['e1489825-752a-4858-b711-8618e0a7bb8a'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:04:58.341331 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] Routers {'e1489825-752a-4858-b711-8618e0a7bb8a'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:58.342901 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['e1489825-752a-4858-b711-8618e0a7bb8a'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:58.398713 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] 10.222.0.48 "PUT /v2.0/routers/e1489825-752a-4858-b711-8618e0a7bb8a/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.9452188 May 31 12:04:58.401708 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40354) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:04:58.444707 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:58.480218 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:58.574038 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] Set router e1489825-752a-4858-b711-8618e0a7bb8a gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:58.614670 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Notify callbacks [] for port, precommit_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:58.632644 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Transition to ACTIVE for port object 68599998-f433-42bf-80ae-c64a075f3712 will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:04:58.645192 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Allocated IP 172.27.1.112 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/2cf42247-1a41-4760-8c77-b8388402439a) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:04:58.660126 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Transition to ACTIVE for port object 68599998-f433-42bf-80ae-c64a075f3712 will not be triggered until provisioned by entity DHCP. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:04:58.667854 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:57Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:58.672135 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2284ef7a-5379-4871-be5b-7e2994050b91 None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:57Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:04:58.765021 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks [] for router, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:58.767335 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699'] for port, after_create {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:58.777455 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:58.877611 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:58.890472 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Network 219aba32-474c-4847-b9d2-5a8ddc5c5369 is already hosted by enough agents. {{(pid=99041) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:04:58.900542 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.scheduler.base_scheduler [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Resources bound (agent type, host, resource id): {{(pid=99041) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:04:58.906441 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Received port after_create {{(pid=99041) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:04:58.974975 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Pushing event updated for resources: {'Port': ['ID=68599998-f433-42bf-80ae-c64a075f3712,revision_number=1']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:58.975180 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T12:04:57Z', 'revision_number': 5, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T12:00:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:58.975351 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:58.987110 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] Checking router: e1489825-752a-4858-b711-8618e0a7bb8a for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:04:59.000383 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] Routers {'e1489825-752a-4858-b711-8618e0a7bb8a'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:04:59.011356 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['e1489825-752a-4858-b711-8618e0a7bb8a'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:59.024040 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.028916 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1890e68f-3cc2-4123-a0c8-8eab3f5ec539 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:59.066038 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Attempting to bind port 68599998-f433-42bf-80ae-c64a075f3712 on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:04:59.066600 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Attempting to bind port 68599998-f433-42bf-80ae-c64a075f3712 by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:04:59.066979 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Attempting to bind port 68599998-f433-42bf-80ae-c64a075f3712 on network 219aba32-474c-4847-b9d2-5a8ddc5c5369 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:04:59.079120 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 077741d4-ebe8-4cb3-9d13-97cd5c09d04c up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:04:59.080105 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.110331 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 4, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:04:59.110707 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Checking segment: {'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:04:59.111118 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.111514 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Bound using segment: {'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:04:59.112147 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Bound port: 68599998-f433-42bf-80ae-c64a075f3712, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:04:59.112382 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.146542 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:04:59.149700 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:04:59.153445 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:04:59.194570 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Pushing event updated for resources: {'Port': ['ID=2cf42247-1a41-4760-8c77-b8388402439a,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:59.198413 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:59.248508 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] For port 68599998-f433-42bf-80ae-c64a075f3712, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:04:59.252132 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] For port 68599998-f433-42bf-80ae-c64a075f3712, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=68599998-f433-42bf-80ae-c64a075f3712,segment=NetworkSegment(9bd3a04f-0739-4430-9c94-924934a7ab52),segment_id=9bd3a04f-0739-4430-9c94-924934a7ab52)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:04:59.264461 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Deleting port 35f9ff48-03b5-48c7-8a4e-da38e484d698 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:04:59.264796 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.282956 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] Set router e1489825-752a-4858-b711-8618e0a7bb8a gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:04:59.327218 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:04:59.332313 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:04:59.332762 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.392079 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.413409 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:59.444305 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] For port 35f9ff48-03b5-48c7-8a4e-da38e484d698, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=35f9ff48-03b5-48c7-8a4e-da38e484d698,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:59.452642 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.547308 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Pushing event updated for resources: {'Port': ['ID=68599998-f433-42bf-80ae-c64a075f3712,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:59.554441 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.143s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:04:59.620997 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Calling delete_port for 35f9ff48-03b5-48c7-8a4e-da38e484d698 owned by network:router_gateway {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:04:59.665425 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.691966 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T12:04:57Z', 'revision_number': 5, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T12:00:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:04:59.705350 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-77e51674-3281-4841-bc2b-5a794cc364b0 None None] The sync data for ids ['e1489825-752a-4858-b711-8618e0a7bb8a'] in dv-cinder-791239-4-iscsi is: [{'id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'name': 'tempest-TestShelveInstance-1085702316-router', 'tenant_id': '00afe2f4c68c431385f2a4b6061b0f72', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134'}], 'enable_snat': True}, 'gw_port_id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T11:59:56Z', 'updated_at': '2021-05-31T12:04:57Z', 'revision_number': 5, 'project_id': '00afe2f4c68c431385f2a4b6061b0f72', 'gw_port': {'id': '35f9ff48-03b5-48c7-8a4e-da38e484d698', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:15:29:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e1489825-752a-4858-b711-8618e0a7bb8a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 521, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.134', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T11:59:57Z', 'updated_at': '2021-05-31T12:00:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:04:59.719532 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:04:59.812207 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:04:59.815987 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:04:59.915696 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:04:59.919915 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:00.019042 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.049028 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.053167 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5ac437c9-4725-4c23-a1d0-51899d469300 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:00.098448 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Pushing event deleted for resources: {'Port': ['ID=35f9ff48-03b5-48c7-8a4e-da38e484d698,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:00.104288 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:00.217639 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.404010 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.406572 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.441233 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-505668', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-507892'] for router, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.555065 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.555721 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Fanout notify agent at l3_agent the message router_deleted on router e1489825-752a-4858-b711-8618e0a7bb8a {{(pid=99040) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 12:05:00.568769 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Notify callbacks [] for router, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.569905 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] 10.222.0.48 "DELETE /v2.0/routers/e1489825-752a-4858-b711-8618e0a7bb8a HTTP/1.1" status: 204 len: 168 time: 2.1659579 May 31 12:05:00.572565 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40362) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:00.634439 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_v2 [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] Deleting subnet 2cdd9ddd-2036-4d15-a3ee-da37da24fae1 {{(pid=99040) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:05:00.638062 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.692884 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3096829'] for subnet, precommit_delete_associations {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.726509 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Router 0e144741-d108-4809-906b-0ffd45f625d3 is scheduled to L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 with binding_index 1 {{(pid=99039) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 31 12:05:00.776770 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:00.956305 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:05:00.967477 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Sync routers for ids ['0e144741-d108-4809-906b-0ffd45f625d3'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:05:00.981905 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Routers {'0e144741-d108-4809-906b-0ffd45f625d3'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:05:00.982688 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0e144741-d108-4809-906b-0ffd45f625d3'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:01.072012 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-39be9234-a434-435d-bba4-392b876de1bf None None] Network 219aba32-474c-4847-b9d2-5a8ddc5c5369 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:05:01.121281 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] Notify callbacks [] for router, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:01.122641 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-59ae9b46-c6e5-47e0-9b4c-f556b3ed69ca admin admin] 10.222.0.48 "POST /v2.0/routers HTTP/1.1" status: 201 len: 923 time: 3.8425703 May 31 12:05:01.125321 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40370) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:01.235201 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2111654', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2831123'] for subnet, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:01.240233 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-828b0abd-9fe3-4ccf-9cdb-ccb0e0be2d83 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:01.278553 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=2cdd9ddd-2036-4d15-a3ee-da37da24fae1,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:01.280407 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:01.286437 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:01.289509 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-18ab9f45-099a-475b-8be7-fc4cb7e17028 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/2cdd9ddd-2036-4d15-a3ee-da37da24fae1 HTTP/1.1" status: 204 len: 168 time: 0.7159481 May 31 12:05:01.290162 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40376) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:01.381398 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-78104b13-a570-4b9f-a8b2-862cd16f2822 None None] Network f1496a40-5ccd-4222-984b-dbffd9fdd474 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:05:01.418514 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:01.429405 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 077741d4-ebe8-4cb3-9d13-97cd5c09d04c up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:05:01.512741 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Set router 0e144741-d108-4809-906b-0ffd45f625d3 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:05:01.522837 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Deleting port 6c20d80c-0f24-4be2-bace-18598335a7ae {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:05:01.523185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:01.636980 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] For port 6c20d80c-0f24-4be2-bace-18598335a7ae, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=6c20d80c-0f24-4be2-bace-18598335a7ae,segment=NetworkSegment(29648ab6-8c7b-47d7-9f64-2b32a50ee3f7),segment_id=29648ab6-8c7b-47d7-9f64-2b32a50ee3f7)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:05:01.637337 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:01.641467 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-fbb817d2-68c5-4463-8b65-021449b98794 None None] Provisioning for port 68599998-f433-42bf-80ae-c64a075f3712 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:05:01.778082 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Calling delete_port for 6c20d80c-0f24-4be2-bace-18598335a7ae owned by network:dhcp {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:05:01.913900 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:01.919653 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-be4afc79-cd2b-4fba-aac2-c037ad555fd4 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:01.979638 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:01.980200 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.069519 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Pushing event deleted for resources: {'Port': ['ID=6c20d80c-0f24-4be2-bace-18598335a7ae,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:02.076503 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] No DVR routers for this DVR port 6c20d80c-0f24-4be2-bace-18598335a7ae on host dv-cinder-791239-4-iscsi {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:05:02.078193 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:02.107715 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.209698 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.218862 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112'}], 'enable_snat': True}, 'gw_port_id': '2cf42247-1a41-4760-8c77-b8388402439a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 3, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port': {'id': '2cf42247-1a41-4760-8c77-b8388402439a', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:b7:9c:01', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:58Z', 'updated_at': '2021-05-31T12:04:58Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:02.223751 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['f1496a40-5ccd-4222-984b-dbffd9fdd474']}} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:02.237671 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Checking router: 0e144741-d108-4809-906b-0ffd45f625d3 for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:05:02.238218 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Updating router 0e144741-d108-4809-906b-0ffd45f625d3 port 2cf42247-1a41-4760-8c77-b8388402439a binding host dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 12:05:02.238695 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7') {'for_net_delete': True} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:02.239207 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '29648ab6-8c7b-47d7-9f64-2b32a50ee3f7') {} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:02.250798 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3096822'] for segment, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.275762 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.276960 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Releasing vlan 365 on physical network physnet1 to pool {{(pid=99040) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:05:02.277360 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2278144'] for segment, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.283041 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-785ca14a-9035-4e70-bfed-6b19f05cb950 None None] DHCP port deletion for f1496a40-5ccd-4222-984b-dbffd9fdd474 request from dv-cinder-791239-4-iscsi {{(pid=99041) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: Traceback (most recent call last): May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: timer() May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: cb(*args, **kw) May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: return func(*args, **kwargs) May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: self._notify() May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: self.callback(batched_events) May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: event.method(event) May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: return f(self, *a, **k) May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._get(url).json() May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: return self.request(url, 'GET', **kwargs) May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:05:02.294693 dv-cinder-791239-4-iscsi neutron-server[99040]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:05:02.307606 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 68599998-f433-42bf-80ae-c64a075f3712 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:05:02.363471 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_common [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Allocated IP 172.20.0.1 (219aba32-474c-4847-b9d2-5a8ddc5c5369/03af805c-fc50-465f-bbbf-187db4b13f93/4a85e41c-d278-4239-b4f9-257929f10c1a) {{(pid=99039) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:05:02.365498 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.475523 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.481477 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b5373c11-0c32-44c9-bc8f-ebbf5d0d0da1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:02.511703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.514675 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Pushing event deleted for resources: {'Network': ['ID=f1496a40-5ccd-4222-984b-dbffd9fdd474,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:02.518287 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:02.533939 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] 10.222.0.48 "DELETE /v2.0/networks/f1496a40-5ccd-4222-984b-dbffd9fdd474 HTTP/1.1" status: 204 len: 168 time: 1.2427003 May 31 12:05:02.549760 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify callbacks [] for port, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.566782 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.provisioning_blocks [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Transition to ACTIVE for port object 4a85e41c-d278-4239-b4f9-257929f10c1a will not be triggered until provisioned by entity DHCP. {{(pid=99039) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:05:02.590544 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 68599998-f433-42bf-80ae-c64a075f3712 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:05:02.606102 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 68599998-f433-42bf-80ae-c64a075f3712 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:02.606481 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.652733 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.657367 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:02.709449 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-84ca4e2b-d368-4f2b-b52a-a4b21281ec8a None None] Provisioning complete for port 68599998-f433-42bf-80ae-c64a075f3712 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:02.709846 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-84ca4e2b-d368-4f2b-b52a-a4b21281ec8a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.719175 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Network 219aba32-474c-4847-b9d2-5a8ddc5c5369 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:05:02.722077 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:05:02.725780 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Received port after_create {{(pid=99039) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:05:02.773471 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Pushing event updated for resources: {'Port': ['ID=4a85e41c-d278-4239-b4f9-257929f10c1a,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:02.779288 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:02.795485 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-501637'] for router_interface, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:02.899357 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.001521 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40386) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:03.048221 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Notify callbacks [] for port, precommit_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.058302 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-23fa243e-e526-4379-8b59-44a67dea407e admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=bbcb901221a14602a1f2f8b198287636&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0526524 May 31 12:05:03.058629 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40392) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:03.076879 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 68599998-f433-42bf-80ae-c64a075f3712, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=68599998-f433-42bf-80ae-c64a075f3712,segment=NetworkSegment(9bd3a04f-0739-4430-9c94-924934a7ab52),segment_id=9bd3a04f-0739-4430-9c94-924934a7ab52)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:05:03.078233 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Transition to ACTIVE for port object 2cf42247-1a41-4760-8c77-b8388402439a will not be triggered until provisioned by entity L2. {{(pid=99041) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:05:03.082892 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-84ca4e2b-d368-4f2b-b52a-a4b21281ec8a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.120931 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-605c716f-34ad-4fe7-9ee8-fcec4ae9fec9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.147025 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-605c716f-34ad-4fe7-9ee8-fcec4ae9fec9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.281243 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.290660 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.291700 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5055f9b3-678e-4847-b40d-783fe291c9f8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:03.296537 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.342599 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-605c716f-34ad-4fe7-9ee8-fcec4ae9fec9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.350836 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:03.353090 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device e3e2dad9-58b2-48c0-8b94-ffde87a6b32a no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:05:03.374745 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-605c716f-34ad-4fe7-9ee8-fcec4ae9fec9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=31834e32-c3e8-4d1f-acda-99841f2d99c8,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:03.377163 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-605c716f-34ad-4fe7-9ee8-fcec4ae9fec9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:03.420177 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-605c716f-34ad-4fe7-9ee8-fcec4ae9fec9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.421272 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-605c716f-34ad-4fe7-9ee8-fcec4ae9fec9 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/31834e32-c3e8-4d1f-acda-99841f2d99c8 HTTP/1.1" status: 204 len: 168 time: 0.3616438 May 31 12:05:03.433006 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: e3e2dad9-58b2-48c0-8b94-ffde87a6b32a {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:05:03.433383 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device e3e2dad9-58b2-48c0-8b94-ffde87a6b32a not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:05:03.473721 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Attempting to bind port 2cf42247-1a41-4760-8c77-b8388402439a on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:05:03.474150 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Attempting to bind port 2cf42247-1a41-4760-8c77-b8388402439a by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:05:03.474525 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Attempting to bind port 2cf42247-1a41-4760-8c77-b8388402439a on network 8a73d753-6759-4d77-a99f-016e97b0a680 {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:05:03.486371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 4, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:05:03.488360 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Checking segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99041) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:05:03.488360 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.488360 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Bound using segment: {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'} {{(pid=99041) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:05:03.488360 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.managers [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Bound port: 2cf42247-1a41-4760-8c77-b8388402439a, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'ec17477a-1229-4e63-8304-a2f0dbd731dd', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680'}}] {{(pid=99041) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:05:03.488360 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.517920 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=68599998-f433-42bf-80ae-c64a075f3712,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:03.523881 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.db.api [None req-84ca4e2b-d368-4f2b-b52a-a4b21281ec8a None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99041) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 31 12:05:03.523881 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_db.api [None req-84ca4e2b-d368-4f2b-b52a-a4b21281ec8a None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 31 12:05:03.593437 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 35f9ff48-03b5-48c7-8a4e-da38e484d698 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:05:03.596411 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify callbacks [] for port, precommit_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.645445 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 35f9ff48-03b5-48c7-8a4e-da38e484d698 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:05:03.645820 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 35f9ff48-03b5-48c7-8a4e-da38e484d698 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:05:03.646223 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 6c20d80c-0f24-4be2-bace-18598335a7ae no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:05:03.669996 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2184939'] for port, after_update {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.686989 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 6c20d80c-0f24-4be2-bace-18598335a7ae {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:05:03.687431 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 6c20d80c-0f24-4be2-bace-18598335a7ae not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:05:03.691716 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d766921c-4c58-4c0a-a8e3-265a90dd1995 None None] Pushing event updated for resources: {'Port': ['ID=2cf42247-1a41-4760-8c77-b8388402439a,revision_number=2']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:03.697503 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-d766921c-4c58-4c0a-a8e3-265a90dd1995 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.405s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:03.698414 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.378s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:03.698555 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:03.700858 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-7db05227-23b5-4780-89de-26ed411e1769 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:03.701475 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] For port 2cf42247-1a41-4760-8c77-b8388402439a, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99041) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:05:03.707706 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] For port 2cf42247-1a41-4760-8c77-b8388402439a, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=2cf42247-1a41-4760-8c77-b8388402439a,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:05:03.783657 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.783912 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-549389', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-501544'] for router_interface, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:03.787473 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Pushing event updated for resources: {'Port': ['ID=4a85e41c-d278-4239-b4f9-257929f10c1a,revision_number=2']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:03.793765 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:03.794051 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:03.862711 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Routers {'0e144741-d108-4809-906b-0ffd45f625d3'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:05:03.863345 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0e144741-d108-4809-906b-0ffd45f625d3'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:03.922002 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d766921c-4c58-4c0a-a8e3-265a90dd1995 None None] Pushing event updated for resources: {'Port': ['ID=2cf42247-1a41-4760-8c77-b8388402439a,revision_number=3']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:03.932676 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-d766921c-4c58-4c0a-a8e3-265a90dd1995 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:03.936863 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Network 219aba32-474c-4847-b9d2-5a8ddc5c5369 is already hosted by enough agents. {{(pid=99039) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:05:03.938115 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.base_scheduler [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Resources bound (agent type, host, resource id): {{(pid=99039) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:05:04.164512 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] Set router 0e144741-d108-4809-906b-0ffd45f625d3 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:05:04.288968 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Router 0e144741-d108-4809-906b-0ffd45f625d3 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99039) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:05:04.356120 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40406) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:04.383193 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-c319b3c9-8280-4f42-b77c-cf11d40d171a None None] Provisioning for port 4a85e41c-d278-4239-b4f9-257929f10c1a completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:05:04.396609 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-c319b3c9-8280-4f42-b77c-cf11d40d171a None None] Provisioning complete for port 4a85e41c-d278-4239-b4f9-257929f10c1a triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:04.396983 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c319b3c9-8280-4f42-b77c-cf11d40d171a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:04.420815 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-1ec35896-694e-4a69-82b6-4975476dde19 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=00afe2f4c68c431385f2a4b6061b0f72&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0609488 May 31 12:05:04.421641 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40410) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:04.483447 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-c319b3c9-8280-4f42-b77c-cf11d40d171a None None] Port 4a85e41c-d278-4239-b4f9-257929f10c1a cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:05:04.494272 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99039) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:05:04.495109 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a8fd33f8-7506-4e0c-9d8a-48f28f62ec56 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:04.499814 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d5083cc6-e97a-451f-ab83-e0d5f542c0fc admin admin] 10.222.0.48 "PUT /v2.0/routers/0e144741-d108-4809-906b-0ffd45f625d3/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.3732822 May 31 12:05:04.532926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a8fd33f8-7506-4e0c-9d8a-48f28f62ec56 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:04.575830 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112'}], 'enable_snat': True}, 'gw_port_id': '2cf42247-1a41-4760-8c77-b8388402439a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:05:02Z', 'revision_number': 4, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port': {'id': '2cf42247-1a41-4760-8c77-b8388402439a', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:b7:9c:01', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:58Z', 'updated_at': '2021-05-31T12:05:03Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '4a85e41c-d278-4239-b4f9-257929f10c1a', 'name': '', 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'mac_address': 'fa:16:3e:7d:58:26', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_interface', 'standard_attr_id': 664, 'fixed_ips': [{'subnet_id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:05:02Z', 'updated_at': '2021-05-31T12:05:03Z', 'revision_number': 2, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'subnets': [{'id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:04.579999 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5d9b2a53-e3ba-457d-b3f7-3e97d2378892 None None] The sync data for ids ['0e144741-d108-4809-906b-0ffd45f625d3'] in dv-cinder-791239-4-iscsi is: [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112'}], 'enable_snat': True}, 'gw_port_id': '2cf42247-1a41-4760-8c77-b8388402439a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:05:02Z', 'revision_number': 4, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port': {'id': '2cf42247-1a41-4760-8c77-b8388402439a', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:b7:9c:01', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:58Z', 'updated_at': '2021-05-31T12:05:03Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '4a85e41c-d278-4239-b4f9-257929f10c1a', 'name': '', 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'mac_address': 'fa:16:3e:7d:58:26', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_interface', 'standard_attr_id': 664, 'fixed_ips': [{'subnet_id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:05:02Z', 'updated_at': '2021-05-31T12:05:03Z', 'revision_number': 2, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'subnets': [{'id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:05:04.668888 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a8fd33f8-7506-4e0c-9d8a-48f28f62ec56 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:04.672757 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6501b731-9a7f-485e-8823-6b49ecd94894 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:04.684525 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a8fd33f8-7506-4e0c-9d8a-48f28f62ec56 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=93a559f8-0bff-4177-ad11-64a4f77793cc,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:04.688242 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a8fd33f8-7506-4e0c-9d8a-48f28f62ec56 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:04.691749 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-a8fd33f8-7506-4e0c-9d8a-48f28f62ec56 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:04.693050 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-a8fd33f8-7506-4e0c-9d8a-48f28f62ec56 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/93a559f8-0bff-4177-ad11-64a4f77793cc HTTP/1.1" status: 204 len: 168 time: 0.2699387 May 31 12:05:05.324665 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40424) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:05.340557 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-2001634369', 'description': 'tempest-secgroup-smoke-2001634369 description', 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:05.340557 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:05.343330 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:security_group and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:05.346198 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:05.355463 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:05.434100 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:05.456764 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:05.460998 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:security_group_rule and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:05.465804 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:05.473725 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:05.561702 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:05.592371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 2cf42247-1a41-4760-8c77-b8388402439a up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:05:05.715067 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:05.719031 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c7942774-21e2-40cb-981f-d25c62fb6a7f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:05.749209 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 2cf42247-1a41-4760-8c77-b8388402439a completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:05:05.754999 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 2cf42247-1a41-4760-8c77-b8388402439a triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:05.760331 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:05.762495 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'SecurityGroup': ['ID=bd77d026-8f31-4ec6-a2ff-f8e6a6857859,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:05.769620 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:05.868385 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:05.874744 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-58c403e5-05c5-422d-9c0c-6bc6476dfb41 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1794 time: 0.5440686 May 31 12:05:05.874744 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40428) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:05.877325 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:05.877992 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.v2.base [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'security_group_rule': {'security_group_id': 'bd77d026-8f31-4ec6-a2ff-f8e6a6857859', 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99039) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:05.883859 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:05.888114 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:security_group_rule and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:05.891044 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:05.894169 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 2cf42247-1a41-4760-8c77-b8388402439a, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=2cf42247-1a41-4760-8c77-b8388402439a,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:05:05.896791 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:05.983359 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:05.984128 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 68599998-f433-42bf-80ae-c64a075f3712 up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:05:06.014629 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-dc713858-f3d7-4377-892d-38c2379446a6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:06.023166 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.088775 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.101821 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=2cf42247-1a41-4760-8c77-b8388402439a,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:06.105414 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:06.114312 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 68599998-f433-42bf-80ae-c64a075f3712 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:06.114691 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.284079 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.291144 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:06.311625 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=bbc60a85-7973-430a-9110-9648529bce16,revision_number=0']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:06.314751 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:06.429502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.430783 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-e5d44921-7771-45db-b7e7-7f06f38ca3ee tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.5571289 May 31 12:05:06.457596 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40434) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:06.457596 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'security_group_rule': {'security_group_id': 'bd77d026-8f31-4ec6-a2ff-f8e6a6857859', 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:06.457596 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:06.457596 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:security_group_rule and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:06.457596 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:06.460012 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:06.541790 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Sync routers for ids ['0e144741-d108-4809-906b-0ffd45f625d3'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:05:06.551159 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Routers {'0e144741-d108-4809-906b-0ffd45f625d3'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:05:06.551781 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0e144741-d108-4809-906b-0ffd45f625d3'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:06.554693 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.616187 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.703102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.705239 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a0fa6a72-9f30-4b9e-a7a1-15097f4dd03f tempest-TestMinimumBasicScenario-1532899900 tempest-TestMinimumBasicScenario-1532899900-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:06.728649 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=31e1f1f4-e632-4b6b-929d-88955ce3d5cf,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:06.731146 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:06.731503 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Set router 0e144741-d108-4809-906b-0ffd45f625d3 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:05:06.800312 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.801356 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-08906666-070d-4029-93b7-da927d8f72e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.3662417 May 31 12:05:06.804098 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40438) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:06.809535 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'security_group_rule': {'security_group_id': 'bd77d026-8f31-4ec6-a2ff-f8e6a6857859', 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:06.815382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:06.819260 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:security_group_rule and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:06.822832 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:06.829018 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:06.921195 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:06.987491 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.075580 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112'}], 'enable_snat': True}, 'gw_port_id': '2cf42247-1a41-4760-8c77-b8388402439a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:05:02Z', 'revision_number': 4, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port': {'id': '2cf42247-1a41-4760-8c77-b8388402439a', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:b7:9c:01', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:58Z', 'updated_at': '2021-05-31T12:05:05Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '4a85e41c-d278-4239-b4f9-257929f10c1a', 'name': '', 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'mac_address': 'fa:16:3e:7d:58:26', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_interface', 'standard_attr_id': 664, 'fixed_ips': [{'subnet_id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:05:02Z', 'updated_at': '2021-05-31T12:05:03Z', 'revision_number': 2, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'subnets': [{'id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:07.092342 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Checking router: 0e144741-d108-4809-906b-0ffd45f625d3 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:05:07.092692 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Updating router 0e144741-d108-4809-906b-0ffd45f625d3 port 4a85e41c-d278-4239-b4f9-257929f10c1a binding host dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 31 12:05:07.114589 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.122220 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-007412c1-d98a-4239-8dd8-6885d492ef67 tempest-AttachVolumeTestJSON-329817016 tempest-AttachVolumeTestJSON-329817016-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:07.141219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=75b2cde2-3e40-4462-bf01-5beb266277aa,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:07.143547 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:07.225869 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.320614 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.323107 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-d6de4ab1-9554-4ee6-aca5-5cfa61ce2b31 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.5168602 May 31 12:05:07.325037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40442) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:07.330045 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'security_group_rule': {'security_group_id': 'bd77d026-8f31-4ec6-a2ff-f8e6a6857859', 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:07.335760 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:07.339876 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:security_group_rule and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:07.342915 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:07.348627 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:07.482449 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.520762 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 2cf42247-1a41-4760-8c77-b8388402439a up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:05:07.554062 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.590482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 2cf42247-1a41-4760-8c77-b8388402439a triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:07.590782 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.623206 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.625294 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:07.646571 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=8566ad9e-df5d-45bf-aa00-7f8eda293fbe,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:07.651259 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:07.688436 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Notify callbacks [] for port, precommit_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.700618 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.provisioning_blocks [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Transition to ACTIVE for port object 4a85e41c-d278-4239-b4f9-257929f10c1a will not be triggered until provisioned by entity L2. {{(pid=99042) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:05:07.714450 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.715661 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-d0dab3cb-60b4-45dd-98ed-e5203ca479de tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.3896976 May 31 12:05:07.718354 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40448) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:07.723442 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'security_group_rule': {'security_group_id': 'bd77d026-8f31-4ec6-a2ff-f8e6a6857859', 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:07.730215 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:07.734023 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:security_group_rule and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:07.737004 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:07.742696 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:07.768052 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.772072 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:07.817365 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.securitygroups_db [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Project 0e2c6e0e5c334c60a47f2dc2d0c79653 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 31 12:05:07.823310 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.902704 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.929656 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Pushing event updated for resources: {'Port': ['ID=4a85e41c-d278-4239-b4f9-257929f10c1a,revision_number=3']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:07.946475 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.947494 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.947955 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.948287 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.948614 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.948939 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.949267 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.949604 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.949952 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.950272 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.950591 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.950904 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99042) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:07.977398 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:07.979055 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.206s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:07.986040 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:08.006141 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Attempting to bind port 4a85e41c-d278-4239-b4f9-257929f10c1a on host dv-cinder-791239-4-iscsi for vnic_type normal with profile {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:05:08.006747 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Attempting to bind port 4a85e41c-d278-4239-b4f9-257929f10c1a by drivers openvswitch,hyperv on host dv-cinder-791239-4-iscsi at level 0 using segments [{'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:05:08.006894 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=b7aa3a3a-1c4d-47b1-94e7-41c56b57bd77,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:08.007115 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Attempting to bind port 4a85e41c-d278-4239-b4f9-257929f10c1a on network 219aba32-474c-4847-b9d2-5a8ddc5c5369 {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:05:08.009147 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:08.013148 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Checking agent: {'id': '896efa02-a2e8-4203-8c35-aa2f328f1281', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-4-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 46, 23), 'started_at': datetime.datetime(2021, 5, 31, 11, 46, 53), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 4, 53), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-4-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:05:08.013528 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Checking segment: {'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99042) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:05:08.023385 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:08.023385 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Bound using segment: {'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'} {{(pid=99042) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:05:08.023385 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.managers [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Bound port: 4a85e41c-d278-4239-b4f9-257929f10c1a, host: dv-cinder-791239-4-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '9bd3a04f-0739-4430-9c94-924934a7ab52', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 370, 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369'}}] {{(pid=99042) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:05:08.023385 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:08.108081 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] For port 4a85e41c-d278-4239-b4f9-257929f10c1a, host dv-cinder-791239-4-iscsi, cleared binding levels {{(pid=99042) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:05:08.110976 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.plugins.ml2.db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] For port 4a85e41c-d278-4239-b4f9-257929f10c1a, host dv-cinder-791239-4-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=4a85e41c-d278-4239-b4f9-257929f10c1a,segment=NetworkSegment(9bd3a04f-0739-4430-9c94-924934a7ab52),segment_id=9bd3a04f-0739-4430-9c94-924934a7ab52)] {{(pid=99042) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:05:08.183863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:08.184907 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-bd45b91c-752f-4458-af96-e778cec47f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.4656291 May 31 12:05:08.187764 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40452) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:08.192916 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'security_group_rule': {'security_group_id': 'bd77d026-8f31-4ec6-a2ff-f8e6a6857859', 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:08.199290 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:08.203065 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:security_group_rule and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:08.205894 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:08.211827 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:08.212392 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron_lib.callbacks.manager [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99042) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:08.218035 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:08.292176 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Routers {'0e144741-d108-4809-906b-0ffd45f625d3'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:05:08.292546 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.securitygroups_db [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Project 0e2c6e0e5c334c60a47f2dc2d0c79653 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 31 12:05:08.292811 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:04:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0e144741-d108-4809-906b-0ffd45f625d3'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:08.299145 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:08.350811 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Pushing event updated for resources: {'Port': ['ID=4a85e41c-d278-4239-b4f9-257929f10c1a,revision_number=4']} {{(pid=99042) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:08.365234 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG oslo_concurrency.lockutils [None req-65a7e14e-6486-490b-9bcb-d76d765b442a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=99042) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:08.372545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:08.465793 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:05:08.468526 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412546'] for security_group_rule, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:08.470642 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:08.491039 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] Set router 0e144741-d108-4809-906b-0ffd45f625d3 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:05:08.491507 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=ff0de91a-bb17-44aa-9409-b1d6d3003b34,revision_number=0']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:08.494029 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:08.628665 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:08.630533 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-6bb3b1c3-a665-4a3d-ba68-4c306ba78f77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.4418128 May 31 12:05:08.862108 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112'}], 'enable_snat': True}, 'gw_port_id': '2cf42247-1a41-4760-8c77-b8388402439a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:05:02Z', 'revision_number': 4, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port': {'id': '2cf42247-1a41-4760-8c77-b8388402439a', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:b7:9c:01', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:58Z', 'updated_at': '2021-05-31T12:05:05Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '4a85e41c-d278-4239-b4f9-257929f10c1a', 'name': '', 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'mac_address': 'fa:16:3e:7d:58:26', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_interface', 'standard_attr_id': 664, 'fixed_ips': [{'subnet_id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:05:02Z', 'updated_at': '2021-05-31T12:05:08Z', 'revision_number': 4, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'subnets': [{'id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:08.866036 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-36bc85ae-77f0-4f7c-9ad4-8453637faa96 None None] The sync data for ids ['0e144741-d108-4809-906b-0ffd45f625d3'] in dv-cinder-791239-4-iscsi is: [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112'}], 'enable_snat': True}, 'gw_port_id': '2cf42247-1a41-4760-8c77-b8388402439a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:05:02Z', 'revision_number': 4, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port': {'id': '2cf42247-1a41-4760-8c77-b8388402439a', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:b7:9c:01', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:58Z', 'updated_at': '2021-05-31T12:05:05Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '4a85e41c-d278-4239-b4f9-257929f10c1a', 'name': '', 'network_id': '219aba32-474c-4847-b9d2-5a8ddc5c5369', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'mac_address': 'fa:16:3e:7d:58:26', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_interface', 'standard_attr_id': 664, 'fixed_ips': [{'subnet_id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:05:02Z', 'updated_at': '2021-05-31T12:05:08Z', 'revision_number': 4, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'subnets': [{'id': '03af805c-fc50-465f-bbbf-187db4b13f93', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:05:09.440148 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:09.520546 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 4a85e41c-d278-4239-b4f9-257929f10c1a up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:05:09.721028 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning for port 4a85e41c-d278-4239-b4f9-257929f10c1a completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:05:09.726266 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 4a85e41c-d278-4239-b4f9-257929f10c1a triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:09.726763 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:09.860745 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:09.894340 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] For port 4a85e41c-d278-4239-b4f9-257929f10c1a, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=4a85e41c-d278-4239-b4f9-257929f10c1a,segment=NetworkSegment(9bd3a04f-0739-4430-9c94-924934a7ab52),segment_id=9bd3a04f-0739-4430-9c94-924934a7ab52)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:05:09.971408 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:09.974095 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-d766921c-4c58-4c0a-a8e3-265a90dd1995 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:10.039299 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Pushing event updated for resources: {'Port': ['ID=4a85e41c-d278-4239-b4f9-257929f10c1a,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:10.043004 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:11.517521 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 4a85e41c-d278-4239-b4f9-257929f10c1a up at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:05:11.599170 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Provisioning complete for port 4a85e41c-d278-4239-b4f9-257929f10c1a triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:11.599549 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:14.783366 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40552) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:14.906286 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:14.920419 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:14.924167 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Usage tracker for resource:security_group_rule and tenant:e6fa656ada694a1eb328e02a1693d6dd is out of sync, need to count used quota {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:14.926938 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99039) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:14.932761 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.quota.driver [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99039) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:15.015513 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:15.125335 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_create {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:15.135101 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-8003c634-cb7b-4436-8cdc-2a784b9df50a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:15.176616 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=e6fa656ada694a1eb328e02a1693d6dd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3921812 May 31 12:05:15.179966 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 40560) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:15.184172 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e8a43525-503d-405b-976f-cb94452c326c,revision_number=1']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:15.198499 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d26b2355-ed32-45e5-b2bf-954b6cfd29c6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.064s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:15.248923 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d747b028-8355-4996-9e62-3347cc7b58f1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:15.279287 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d747b028-8355-4996-9e62-3347cc7b58f1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:15.434302 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d747b028-8355-4996-9e62-3347cc7b58f1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:15.438507 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fdaae31c-37ad-49ce-8250-bd2e551ebc3b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:15.451492 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d747b028-8355-4996-9e62-3347cc7b58f1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e8a43525-503d-405b-976f-cb94452c326c,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:15.455464 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-d747b028-8355-4996-9e62-3347cc7b58f1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:15.458917 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-d747b028-8355-4996-9e62-3347cc7b58f1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:15.464672 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-d747b028-8355-4996-9e62-3347cc7b58f1 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/e8a43525-503d-405b-976f-cb94452c326c HTTP/1.1" status: 204 len: 168 time: 0.2791913 May 31 12:05:20.908371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1d846bd4-a7d7-49b5-a2f6-88cb2bfe0f6d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:23.354548 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:23.456821 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ff87e7d5-237c-472b-926d-d671280bfb36 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:24.593738 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:26.148057 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:26.580978 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-78428fb4-7ced-4eeb-9477-2075a8983b11 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:security_group {{(pid=99039) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:05:26.667815 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.quota.resource [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-78428fb4-7ced-4eeb-9477-2075a8983b11 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:security_group {{(pid=99039) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:05:26.669001 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-78428fb4-7ced-4eeb-9477-2075a8983b11 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-2001634369&fields=id&tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 267 time: 0.1269312 May 31 12:05:26.814300 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-b3390fb4-bab1-4d2f-9da4-c7d394c2fd64 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:26.815685 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-b3390fb4-bab1-4d2f-9da4-c7d394c2fd64 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 811 time: 0.1401010 May 31 12:05:26.830396 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-78c0f84c-ca2b-4da5-a2b2-d0e42034d9c3 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/quotas/0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 375 time: 0.0093133 May 31 12:05:26.914239 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-72ba54a4-539f-453c-9a0c-881c95bc0d36 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0790105 May 31 12:05:26.929833 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-6da33e1c-fec4-4b2d-820c-3fcd3563807b service neutron] http://10.222.0.48:9696/v2.0/extensions returned with HTTP 200 {{(pid=99039) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 31 12:05:26.930760 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-6da33e1c-fec4-4b2d-820c-3fcd3563807b service neutron] 10.222.0.48 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0050418 May 31 12:05:27.061843 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-170dac1f-1da3-42db-bf6c-905eb4776793 service neutron] 10.222.0.48 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1252296 May 31 12:05:27.191837 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-76f84928-6fb3-4138-ad9f-52359d0ea6c9 service neutron] 10.222.0.48 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1241531 May 31 12:05:27.877872 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3f49a027-8a73-439d-8358-35ccf6c62155 req-ef8be69f-468e-425e-ac9d-4c1081d520a8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:05:27.994769 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3f49a027-8a73-439d-8358-35ccf6c62155 req-ef8be69f-468e-425e-ac9d-4c1081d520a8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:05:27.995963 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3f49a027-8a73-439d-8358-35ccf6c62155 req-ef8be69f-468e-425e-ac9d-4c1081d520a8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 210 time: 0.1597629 May 31 12:05:29.143513 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-33f35d29-763c-47bf-9a43-072e3db742b7 req-92cd783e-62ae-467d-871a-597606c1474d tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 210 time: 0.0424981 May 31 12:05:30.348287 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-2d56e6ad-7d86-4edf-b035-539df1bb20e9 req-5cafe6fc-3278-46db-b7b8-9481cdb717aa tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 210 time: 0.0590401 May 31 12:05:30.408798 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-81c90a26-ea6e-44d7-bd1a-c676f41a147a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:30.409491 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-81c90a26-ea6e-44d7-bd1a-c676f41a147a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:network {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:05:30.468296 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-81c90a26-ea6e-44d7-bd1a-c676f41a147a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:network {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:05:30.469443 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-81c90a26-ea6e-44d7-bd1a-c676f41a147a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 811 time: 0.2552645 May 31 12:05:30.518178 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-58854963-3f3f-4374-a88c-7b1c3010eb6c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:security_group {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:05:30.530160 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-58854963-3f3f-4374-a88c-7b1c3010eb6c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:security_group {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:05:30.531271 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-58854963-3f3f-4374-a88c-7b1c3010eb6c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 380 time: 0.0538208 May 31 12:05:30.541130 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'port': {'device_id': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'admin_state_up': True, 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'security_groups': ['bd77d026-8f31-4ec6-a2ff-f8e6a6857859']}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:30.676059 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:30.686253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:30.748121 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Made reservation on behalf of 0e2c6e0e5c334c60a47f2dc2d0c79653 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:05:30.751794 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:30.918045 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Allocated IP 172.20.0.3 (8a4efbb8-aa93-49be-a073-41ce66e25d2f/4a96f417-9ce3-448a-9f7f-529c69a0ace7/15ed134d-debb-4ac0-8153-a8ad4f7f255e) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:05:31.102345 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:31.117422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Transition to ACTIVE for port object 15ed134d-debb-4ac0-8153-a8ad4f7f255e will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:05:31.211458 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:31.217382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ac5b6dee-f092-484b-9464-21e1c157cbd6 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:31.379636 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:05:31.382870 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:05:31.392703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=15ed134d-debb-4ac0-8153-a8ad4f7f255e,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:31.395379 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:05:31.453470 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.236s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:31.471821 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:31.472697 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:31.566233 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.0289776 May 31 12:05:31.572324 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-334b2b24-c9a9-482e-9a83-2e1ab0e96955 req-0366bd8b-e8e4-4d2e-a616-fb0935f1503c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:31.572324 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-334b2b24-c9a9-482e-9a83-2e1ab0e96955 req-0366bd8b-e8e4-4d2e-a616-fb0935f1503c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 931 time: 0.0954633 May 31 12:05:31.578221 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Request body: {'port': {'device_id': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:31.619549 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-334b2b24-c9a9-482e-9a83-2e1ab0e96955 req-35062085-f3d7-48de-a8e0-e9fc4aa67cf1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0440888 May 31 12:05:31.679637 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-d69ee98b-61d3-4ffc-878e-24944b52158a None None] Provisioning for port 15ed134d-debb-4ac0-8153-a8ad4f7f255e completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:05:31.685147 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-d69ee98b-61d3-4ffc-878e-24944b52158a None None] Provisioning complete for port 15ed134d-debb-4ac0-8153-a8ad4f7f255e triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:31.685540 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-d69ee98b-61d3-4ffc-878e-24944b52158a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:31.707058 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:31.738615 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-d69ee98b-61d3-4ffc-878e-24944b52158a None None] Port 15ed134d-debb-4ac0-8153-a8ad4f7f255e cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:05:32.043062 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 15ed134d-debb-4ac0-8153-a8ad4f7f255e {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:05:32.050150 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:32.063679 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Transition to ACTIVE for port object 15ed134d-debb-4ac0-8153-a8ad4f7f255e will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:05:32.159307 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:32.163959 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-427627ee-8c04-4f68-aa47-e4b097ec7eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:32.324791 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Pushing event updated for resources: {'Port': ['ID=15ed134d-debb-4ac0-8153-a8ad4f7f255e,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:32.332220 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.168s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:32.398409 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Attempting to bind port 15ed134d-debb-4ac0-8153-a8ad4f7f255e on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:05:32.399091 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Attempting to bind port 15ed134d-debb-4ac0-8153-a8ad4f7f255e by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:05:32.399091 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Attempting to bind port 15ed134d-debb-4ac0-8153-a8ad4f7f255e on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:05:32.404197 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Port 15ed134d-debb-4ac0-8153-a8ad4f7f255e on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 12:05:32.404511 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Attempting to bind port 15ed134d-debb-4ac0-8153-a8ad4f7f255e on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:05:32.409522 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 5, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:05:32.409877 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Checking segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:05:32.410251 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Bound using segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:05:32.410668 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Bound port: 15ed134d-debb-4ac0-8153-a8ad4f7f255e, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:05:32.411010 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:32.502808 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] For port 15ed134d-debb-4ac0-8153-a8ad4f7f255e, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:05:32.505860 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] For port 15ed134d-debb-4ac0-8153-a8ad4f7f255e, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:05:32.572298 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:32.576321 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:32.633458 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:32.634557 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:32.636604 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] 10.222.0.47 "PUT /v2.0/ports/15ed134d-debb-4ac0-8153-a8ad4f7f255e HTTP/1.1" status: 200 len: 1093 time: 1.0627117 May 31 12:05:32.637102 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'tag': '15ed134d-debb-4ac0-8153-a8ad4f7f255e'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:05:32.639386 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-6e0395ed-f79b-4651-ad09-3de95b9a13c5" -d '{"events": [{"name": "network-changed", "server_uuid": "58d18350-b3dd-4395-aea6-39d797cb95a1", "tag": "15ed134d-debb-4ac0-8153-a8ad4f7f255e"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:05:32.659333 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Pushing event updated for resources: {'Port': ['ID=15ed134d-debb-4ac0-8153-a8ad4f7f255e,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:32.662762 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0398eac0-0c5b-401e-9293-b15664ce8bcd service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:32.830152 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-73956384-d52d-4c65-b385-ec036ce2be41 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:32.831653 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-73956384-d52d-4c65-b385-ec036ce2be41 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 1096 time: 0.0886414 May 31 12:05:32.839254 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ecf03fb3-0708-4e9f-9427-f819e005c67f req-d93bf69f-9118-4a33-876d-e76db8c9c554 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:32.839781 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:05:32 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-bd23407c-3849-4887-b2b4-1ba406cc3b27 x-openstack-request-id: req-bd23407c-3849-4887-b2b4-1ba406cc3b27 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:05:32.840418 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "58d18350-b3dd-4395-aea6-39d797cb95a1", "tag": "15ed134d-debb-4ac0-8153-a8ad4f7f255e", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:05:32.840724 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-bd23407c-3849-4887-b2b4-1ba406cc3b27 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:05:32.841150 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ecf03fb3-0708-4e9f-9427-f819e005c67f req-d93bf69f-9118-4a33-876d-e76db8c9c554 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 943 time: 0.0918610 May 31 12:05:32.841289 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'tag': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'status': 'completed', 'code': 200} May 31 12:05:32.890549 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-ecf03fb3-0708-4e9f-9427-f819e005c67f req-705d65ee-b1dd-49c8-bf02-99c40ef28dd2 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0412719 May 31 12:05:32.905138 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-812269ce-a18a-494e-ad58-81bb20863d56 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e HTTP/1.1" status: 200 len: 217 time: 0.0630252 May 31 12:05:32.971095 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0ba7e5b4-afd7-49f6-a94b-08ecdee0f85d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:32.972512 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-0ba7e5b4-afd7-49f6-a94b-08ecdee0f85d service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0603118 May 31 12:05:33.047361 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-05ac1ec6-687e-4c18-accd-314180d9ab56 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:33.048797 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-05ac1ec6-687e-4c18-accd-314180d9ab56 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0697489 May 31 12:05:33.181677 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-4aa44e49-3d76-4354-8a3c-58e9269249b0 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1253793 May 31 12:05:33.311357 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-38518e00-e61b-4dac-b374-7aba3f6f12f8 req-7d3b4ecf-2a88-4f4e-8756-dc9c126099a8 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1241481 May 31 12:05:33.499543 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-542bfb14-084a-4034-84ae-8c73bcf58de0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:33.500974 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-542bfb14-084a-4034-84ae-8c73bcf58de0 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 1096 time: 0.0698597 May 31 12:05:33.726048 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-c76feea4-f635-41d9-9f95-b89c536e2f01 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:33.727389 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-c76feea4-f635-41d9-9f95-b89c536e2f01 service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.2193530 May 31 12:05:33.794202 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-3f527e81-ae8e-4d75-b856-a9d09e16072b service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e HTTP/1.1" status: 200 len: 217 time: 0.0598032 May 31 12:05:33.857964 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-ce9ee275-fd22-4d3c-b7fe-8c8048181536 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:33.859311 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-ce9ee275-fd22-4d3c-b7fe-8c8048181536 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0582991 May 31 12:05:33.934663 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-626951a3-8f51-4de5-9265-b6450a9322cc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:33.936117 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-626951a3-8f51-4de5-9265-b6450a9322cc service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0692008 May 31 12:05:34.122949 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-d1461b86-6190-4143-8ca9-28bfd7a23c1a service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1796265 May 31 12:05:34.138500 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ebcb924-7474-459d-ade7-bb54428758e4 req-e3dd7655-064a-4973-be0c-8487687e169a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:34.140861 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3ebcb924-7474-459d-ade7-bb54428758e4 req-e3dd7655-064a-4973-be0c-8487687e169a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:05:34.271147 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6e0395ed-f79b-4651-ad09-3de95b9a13c5 req-fb3ffce5-e846-42c8-96e5-826ba0a22ad0 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1406279 May 31 12:05:34.291579 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-3ebcb924-7474-459d-ade7-bb54428758e4 req-e3dd7655-064a-4973-be0c-8487687e169a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:05:34.296117 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3ebcb924-7474-459d-ade7-bb54428758e4 req-e3dd7655-064a-4973-be0c-8487687e169a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 943 time: 0.2807562 May 31 12:05:34.335643 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3ebcb924-7474-459d-ade7-bb54428758e4 req-cb6820da-a727-4595-9562-ba53a73f047e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0374079 May 31 12:05:35.627964 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9fa023c-51ef-4ad8-80c8-6e5cb3178f47 req-c7140e1d-b7ee-40b2-bdd1-4c4da25ae947 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:35.629300 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e9fa023c-51ef-4ad8-80c8-6e5cb3178f47 req-c7140e1d-b7ee-40b2-bdd1-4c4da25ae947 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 943 time: 0.1903951 May 31 12:05:35.674484 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e9fa023c-51ef-4ad8-80c8-6e5cb3178f47 req-2f8391b9-3269-4566-aa9b-834d670d0955 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391021 May 31 12:05:36.852330 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3efa6e73-bd51-4710-82c8-7d0d4e8dfad9 req-d6264248-a4c8-4f2e-bc1d-965873ecfcbf tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:36.854243 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3efa6e73-bd51-4710-82c8-7d0d4e8dfad9 req-d6264248-a4c8-4f2e-bc1d-965873ecfcbf tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 943 time: 0.0784118 May 31 12:05:36.901112 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3efa6e73-bd51-4710-82c8-7d0d4e8dfad9 req-58e66850-6cb8-4a9f-9944-2d428fd80fbf tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0412714 May 31 12:05:38.097807 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8a4ae3a-ed9f-4688-890b-40e561f338e9 req-76d1c090-c42f-4d1d-97fe-36631e83161a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:38.099197 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b8a4ae3a-ed9f-4688-890b-40e561f338e9 req-76d1c090-c42f-4d1d-97fe-36631e83161a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 943 time: 0.0867183 May 31 12:05:38.144835 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-b8a4ae3a-ed9f-4688-890b-40e561f338e9 req-21166a4c-f92c-4f29-a442-3c437f4842a3 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0394812 May 31 12:05:39.328475 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f29b99b8-9910-450d-b59b-fd882bee9676 req-7c609d16-3e47-4871-a08b-ed856dcd39aa tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:39.329933 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f29b99b8-9910-450d-b59b-fd882bee9676 req-7c609d16-3e47-4871-a08b-ed856dcd39aa tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 943 time: 0.0795431 May 31 12:05:39.374018 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f29b99b8-9910-450d-b59b-fd882bee9676 req-1d03ec66-095c-463f-9dc3-5af15336494e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0383868 May 31 12:05:39.388239 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:40.559276 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87a5c877-db89-438e-a6fa-1798dde74e1f req-4d6cfb6d-4e35-4257-ad2e-f94c75ac19b4 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:40.560721 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-87a5c877-db89-438e-a6fa-1798dde74e1f req-4d6cfb6d-4e35-4257-ad2e-f94c75ac19b4 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 943 time: 0.0844517 May 31 12:05:40.604770 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-87a5c877-db89-438e-a6fa-1798dde74e1f req-b6fa0477-1324-4fb3-8cb8-346ab1e87cff tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0384550 May 31 12:05:41.743766 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 15ed134d-debb-4ac0-8153-a8ad4f7f255e details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:05:41.789679 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9cc8d82e-d310-4d08-b7f7-18fe1f978c14 req-c520fe9e-cce3-49a3-8c23-080323586916 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:41.791225 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9cc8d82e-d310-4d08-b7f7-18fe1f978c14 req-c520fe9e-cce3-49a3-8c23-080323586916 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 943 time: 0.0835207 May 31 12:05:41.838014 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9cc8d82e-d310-4d08-b7f7-18fe1f978c14 req-c69511dd-fc4b-4084-b1ef-f70783c28221 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0410504 May 31 12:05:41.876462 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 15ed134d-debb-4ac0-8153-a8ad4f7f255e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:05:41.890045 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'port_id': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'mac_address': 'fa:16:3e:2e:6c:3e', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 362, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:05:41.947882 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:41.964023 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 15ed134d-debb-4ac0-8153-a8ad4f7f255e {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:05:41.979291 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 15ed134d-debb-4ac0-8153-a8ad4f7f255e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:05:42.062075 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:42.063696 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:42.129024 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=15ed134d-debb-4ac0-8153-a8ad4f7f255e,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:42.132119 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:42.265696 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['15ed134d-debb-4ac0-8153-a8ad4f7f255e'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:05:43.020763 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87e88d82-3ba6-4b12-b52b-abf9890fd864 req-b46db989-06a3-427c-891e-0c29fb1c9ccd tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:43.025216 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-87e88d82-3ba6-4b12-b52b-abf9890fd864 req-b46db989-06a3-427c-891e-0c29fb1c9ccd tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 944 time: 0.0824082 May 31 12:05:43.066684 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-87e88d82-3ba6-4b12-b52b-abf9890fd864 req-69e25b96-f760-4c7b-8370-5bc144561eae tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0379276 May 31 12:05:43.759206 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 15ed134d-debb-4ac0-8153-a8ad4f7f255e up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:05:43.902353 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 15ed134d-debb-4ac0-8153-a8ad4f7f255e completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:05:43.907258 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 15ed134d-debb-4ac0-8153-a8ad4f7f255e triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:05:43.907633 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:44.027333 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:44.053767 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '15ed134d-debb-4ac0-8153-a8ad4f7f255e'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:05:44.056101 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-999bad52-5271-423b-9ef2-2e261772b2be" -d '{"events": [{"server_uuid": "58d18350-b3dd-4395-aea6-39d797cb95a1", "name": "network-vif-plugged", "status": "completed", "tag": "15ed134d-debb-4ac0-8153-a8ad4f7f255e"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:05:44.064133 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 15ed134d-debb-4ac0-8153-a8ad4f7f255e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:05:44.113536 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:05:44 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-760e475d-4894-4da0-b329-ad100ed9c0dd x-openstack-request-id: req-760e475d-4894-4da0-b329-ad100ed9c0dd {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:05:44.113864 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "58d18350-b3dd-4395-aea6-39d797cb95a1", "name": "network-vif-plugged", "status": "completed", "tag": "15ed134d-debb-4ac0-8153-a8ad4f7f255e", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:05:44.114121 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-760e475d-4894-4da0-b329-ad100ed9c0dd {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:05:44.114516 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'code': 200} May 31 12:05:44.176081 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:44.178105 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:44.244738 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=15ed134d-debb-4ac0-8153-a8ad4f7f255e,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:44.247791 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:44.256956 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb0a339b-0e9f-411d-941b-0a7efc4fa98b req-b6eb6385-1338-48ba-84af-2ac10142bf02 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:44.258979 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bb0a339b-0e9f-411d-941b-0a7efc4fa98b req-b6eb6385-1338-48ba-84af-2ac10142bf02 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 945 time: 0.0829968 May 31 12:05:44.307698 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-bb0a339b-0e9f-411d-941b-0a7efc4fa98b req-8e0e912c-f98f-42b6-95b4-1ef1241c3981 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0419247 May 31 12:05:44.534219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1fcedbc1-a301-429c-bc9e-ee2bcf0bb1e9 req-0c6c5148-cec9-44d1-a0e7-1d4cdb1caf47 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:44.535676 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1fcedbc1-a301-429c-bc9e-ee2bcf0bb1e9 req-0c6c5148-cec9-44d1-a0e7-1d4cdb1caf47 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 1098 time: 0.0696614 May 31 12:05:44.576120 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1fcedbc1-a301-429c-bc9e-ee2bcf0bb1e9 req-2547c814-7a46-4a74-bcac-77230e2a5101 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0340109 May 31 12:05:45.465926 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:05:45.512269 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ff249320-e07a-4740-9a77-e55e4b61a219 req-6eb83497-b8c6-474c-aa36-73eafb995feb tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:45.513796 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ff249320-e07a-4740-9a77-e55e4b61a219 req-6eb83497-b8c6-474c-aa36-73eafb995feb tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 945 time: 0.0867071 May 31 12:05:45.559296 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ff249320-e07a-4740-9a77-e55e4b61a219 req-6eacf8a8-aef0-409d-8469-4e92fdf2fac5 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0397379 May 31 12:05:46.745234 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0a69a60-f58e-4483-9c3b-aefd4448395c req-4de2d0e4-be4a-4a91-b7e7-06efb03e9e52 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:46.746676 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e0a69a60-f58e-4483-9c3b-aefd4448395c req-4de2d0e4-be4a-4a91-b7e7-06efb03e9e52 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 945 time: 0.0850053 May 31 12:05:46.790508 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-e0a69a60-f58e-4483-9c3b-aefd4448395c req-e8001730-1ee5-4351-b8d9-2f5023badd67 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0374923 May 31 12:05:46.971711 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5209481d-bffe-4c29-9101-95b78661323f req-72aef1a3-4caa-4fb2-b2cd-01020f50819e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:46.973129 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5209481d-bffe-4c29-9101-95b78661323f req-72aef1a3-4caa-4fb2-b2cd-01020f50819e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 945 time: 0.0778379 May 31 12:05:47.016878 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-5209481d-bffe-4c29-9101-95b78661323f req-0205c752-83e2-4354-bb81-0981a2003244 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0381348 May 31 12:05:47.039100 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40666) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:47.276503 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-004f6597-72d7-4fe3-bca0-6c39f9323d53 tempest-TestVolumeBootPattern-1447201260 tempest-TestVolumeBootPattern-1447201260-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:47.278086 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-004f6597-72d7-4fe3-bca0-6c39f9323d53 tempest-TestVolumeBootPattern-1447201260 tempest-TestVolumeBootPattern-1447201260-project-admin] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 1093 time: 0.2379715 May 31 12:05:47.280818 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40672) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:05:47.285672 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'port_id': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address': '172.20.0.3'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:05:47.292206 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:05:47.296056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:floatingip and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:05:47.298816 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:05:47.304394 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:05:47.382211 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:47.524666 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:47.749804 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Allocated IP 172.27.1.79 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/6130415e-0d67-42e4-9395-9bb035dddeea) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:05:47.892928 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:47.987831 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:47.992413 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ff222e6c-53ab-4262-b75d-227178017f56 tempest-AttachVolumeShelveTestJSON-1327742719 tempest-AttachVolumeShelveTestJSON-1327742719-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:48.052520 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:05:48.055955 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:05:48.059719 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:05:48.096062 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=6130415e-0d67-42e4-9395-9bb035dddeea,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:48.112196 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.112573 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.112939 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.113298 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.113680 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.118871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.118871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.118871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.118871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.118871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.118871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.118871 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:05:48.121257 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:48.487608 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:48.674947 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:48.873470 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:48.930931 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:48.935418 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-a3c5bcaf-87ea-41f7-9b5b-29fb9aefec74 tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:05:49.040185 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=6130415e-0d67-42e4-9395-9bb035dddeea,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:05:49.044050 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:05:49.050538 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:49.218359 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Floating IP e64c4829-1a2c-4e4b-832d-03e4b32f81f4 associated. External IP: 172.27.1.79, port: 15ed134d-debb-4ac0-8153-a8ad4f7f255e. May 31 12:05:49.660219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:05:49.844113 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:05:49.853278 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:05:49.862397 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:05:49.863065 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:05:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:49.941903 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:50.009657 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-1014a350-e139-4913-9a17-8a95cd1bb50e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 2.7151527 May 31 12:05:50.009657 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'tag': '15ed134d-debb-4ac0-8153-a8ad4f7f255e'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:05:50.009657 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c" -d '{"events": [{"name": "network-changed", "server_uuid": "58d18350-b3dd-4395-aea6-39d797cb95a1", "tag": "15ed134d-debb-4ac0-8153-a8ad4f7f255e"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:05:50.041322 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:05:50.041496 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:05:50 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-6a946cc9-d6f9-4bff-9035-acbc377ff0c1 x-openstack-request-id: req-6a946cc9-d6f9-4bff-9035-acbc377ff0c1 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:05:50.042219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "58d18350-b3dd-4395-aea6-39d797cb95a1", "tag": "15ed134d-debb-4ac0-8153-a8ad4f7f255e", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:05:50.042219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-6a946cc9-d6f9-4bff-9035-acbc377ff0c1 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:05:50.042512 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'tag': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'status': 'completed', 'code': 200} May 31 12:05:50.137246 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-a8bc809a-3ffe-4b7f-bd6d-902d80e2ceb6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:50.138829 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-a8bc809a-3ffe-4b7f-bd6d-902d80e2ceb6 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 1098 time: 0.0745935 May 31 12:05:50.268737 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-d3b97c47-3642-42a0-aab9-80fa34126aaf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:50.270210 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-d3b97c47-3642-42a0-aab9-80fa34126aaf service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.1249628 May 31 12:05:50.370612 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-4df6a65a-ddcc-413d-9624-6e883c778895 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e HTTP/1.1" status: 200 len: 998 time: 0.0923460 May 31 12:05:50.386452 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'e64c4829-1a2c-4e4b-832d-03e4b32f81f4', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.79', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 681, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:2e:6c:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:05:48Z', 'updated_at': '2021-05-31T12:05:48Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:50.390348 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:05:50.398661 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:05:50.399973 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:05:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:50.435094 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-a2c995f6-79e2-42b7-9268-460785873fe7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:50.436531 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-a2c995f6-79e2-42b7-9268-460785873fe7 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0602043 May 31 12:05:50.514378 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-494c0e08-6581-49ed-b288-e10590791965 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:05:50.515876 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-494c0e08-6581-49ed-b288-e10590791965 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0735886 May 31 12:05:50.576050 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:05:50.666423 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-7f97e40a-4539-44c7-8a19-344944e383ba service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1390350 May 31 12:05:50.797332 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f03ce3b0-c197-4bd7-bc0d-9bf745dca00c req-00c43229-d21f-47c9-be6c-5ed6e3f0e2d5 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1238832 May 31 12:05:50.936630 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'e64c4829-1a2c-4e4b-832d-03e4b32f81f4', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.79', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 681, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:2e:6c:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:05:48Z', 'updated_at': '2021-05-31T12:05:48Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:05:50.940529 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b6a0e957-5f23-4bf9-9dff-bcdf7eb023df None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': 'e64c4829-1a2c-4e4b-832d-03e4b32f81f4', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.79', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 681, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:2e:6c:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:05:48Z', 'updated_at': '2021-05-31T12:05:48Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:05:50.964935 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-995029fc-f9a5-4599-a17a-ce6a66c9a20a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:51.113022 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-aa7218fa-6e67-4e22-a1a8-9fa99a338d7a None None] New status for floating IP e64c4829-1a2c-4e4b-832d-03e4b32f81f4: ACTIVE {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:05:53.370033 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:53.461832 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-d52257de-34a5-43d0-9030-498744e17b1a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:54.633163 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:05:56.170250 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:09.393065 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:20.895664 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-39ee17a9-643c-4757-9b0f-563574a57025 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:22.464761 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:06:23.346684 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:23.449590 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-4ae698b9-57fc-46fa-9d5f-dac95294315f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:24.627349 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:26.106945 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:39.403820 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:45.369313 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-cf544f8a-f8be-44c7-a4f9-e463e294aeb4 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0734508 May 31 12:06:45.505545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-323db73c-2145-4811-9bfd-09d20926c326 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:06:45.507316 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-323db73c-2145-4811-9bfd-09d20926c326 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 1098 time: 0.0700343 May 31 12:06:45.657683 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-16eff451-9643-4c2c-9060-d50bb13be1d8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:06:45.659494 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-16eff451-9643-4c2c-9060-d50bb13be1d8 service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.1339276 May 31 12:06:45.847229 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-89999253-d504-4347-9506-42657c3d715e service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e HTTP/1.1" status: 200 len: 1000 time: 0.1816959 May 31 12:06:45.912455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-9a0e4d96-d593-41fc-ab72-db1a76b09a70 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:06:45.913846 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-9a0e4d96-d593-41fc-ab72-db1a76b09a70 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0603306 May 31 12:06:45.989316 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-6353db80-f520-4e43-b76f-64538cefef3f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:06:45.990845 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-6353db80-f520-4e43-b76f-64538cefef3f service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0711412 May 31 12:06:46.120931 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-9078caaa-dca6-49d8-ad3e-cefcd2fcfcaf service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1235242 May 31 12:06:46.260189 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1026ab40-ef86-44d8-9734-e3f2c2ca9f6b req-ac99f860-d6df-4b93-bd72-0e3f155dd81c service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1336012 May 31 12:06:50.889092 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f238bfef-e3fd-41a9-8397-70fe622256ce None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:53.351665 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:53.449126 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-61875313-cbc6-4c8f-aff2-8fa36e425653 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:54.626071 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:56.150633 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:06:59.463211 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:07:09.402570 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:20.893353 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1ad765d1-ae20-48ac-b140-843785c3ec89 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:23.354562 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:23.446403 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b1aa9c93-4e83-415a-b092-5e9de791d1bc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:24.657379 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:26.161664 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:36.462398 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:07:39.445429 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:40.067414 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-54907675-7375-494f-8fef-509b19418452 req-186bc3ba-ccc1-42b6-a5b6-8d41521f7476 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:40.068886 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-54907675-7375-494f-8fef-509b19418452 req-186bc3ba-ccc1-42b6-a5b6-8d41521f7476 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 945 time: 0.0854807 May 31 12:07:40.113354 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-54907675-7375-494f-8fef-509b19418452 req-e155c758-751d-4bb2-83d7-52a88ff47b19 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0387812 May 31 12:07:41.298661 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-edcd8ab8-7fa2-4bfd-8f13-4bbf1dbfd9c2 req-66d39521-1b9d-4607-b7b1-a826996e7723 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:41.300084 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-edcd8ab8-7fa2-4bfd-8f13-4bbf1dbfd9c2 req-66d39521-1b9d-4607-b7b1-a826996e7723 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 945 time: 0.0823288 May 31 12:07:41.352132 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-edcd8ab8-7fa2-4bfd-8f13-4bbf1dbfd9c2 req-bcd02e90-8582-4ba7-b75b-e1dd27506138 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0464294 May 31 12:07:41.481645 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-93fd6eef-5f7f-4e57-a472-61570a271046 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:41.482959 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-93fd6eef-5f7f-4e57-a472-61570a271046 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 945 time: 0.0883076 May 31 12:07:41.617116 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Deleting port 15ed134d-debb-4ac0-8153-a8ad4f7f255e {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:07:41.617449 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:41.748005 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] For port 15ed134d-debb-4ac0-8153-a8ad4f7f255e, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=15ed134d-debb-4ac0-8153-a8ad4f7f255e,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:07:41.748270 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:42.030036 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:42.030315 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:42.030626 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Floating IP e64c4829-1a2c-4e4b-832d-03e4b32f81f4 disassociated. External IP: 172.27.1.79, port: 15ed134d-debb-4ac0-8153-a8ad4f7f255e. May 31 12:07:42.030959 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Calling delete_port for 15ed134d-debb-4ac0-8153-a8ad4f7f255e owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:07:42.193099 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:42.199306 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e980043e-2656-4568-94eb-bcb140342d0e tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:07:42.250032 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event deleted for resources: {'Port': ['ID=15ed134d-debb-4ac0-8153-a8ad4f7f255e,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:07:42.261546 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.261938 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.262298 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.262653 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.263004 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.263350 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.263694 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.264036 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.264380 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.264721 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.265061 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.265395 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:07:42.269036 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:07:42.505365 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-27efcf08-eae4-4aca-9a6f-620cf9294179 req-de15f25a-6212-4f99-b2e1-eb5f39beed18 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 210 time: 0.0448189 May 31 12:07:42.544571 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:07:42.729736 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:07:42.738949 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:07:42.748673 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:07:42.749288 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:07:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:07:42.803237 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] No DVR routers for this DVR port 15ed134d-debb-4ac0-8153-a8ad4f7f255e on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:07:42.806600 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:42.810675 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'name': 'network-vif-deleted', 'tag': '15ed134d-debb-4ac0-8153-a8ad4f7f255e'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:07:42.812915 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-9cd06bdc-0977-4ef7-9858-cd60354d0629" -d '{"events": [{"server_uuid": "58d18350-b3dd-4395-aea6-39d797cb95a1", "name": "network-vif-deleted", "tag": "15ed134d-debb-4ac0-8153-a8ad4f7f255e"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:07:42.861067 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:07:42 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-51307b58-feff-41bb-95b8-523891b3aafb x-openstack-request-id: req-51307b58-feff-41bb-95b8-523891b3aafb {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:07:42.861067 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "58d18350-b3dd-4395-aea6-39d797cb95a1", "name": "network-vif-deleted", "tag": "15ed134d-debb-4ac0-8153-a8ad4f7f255e", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:07:42.861067 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-51307b58-feff-41bb-95b8-523891b3aafb {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:07:42.861402 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '58d18350-b3dd-4395-aea6-39d797cb95a1', 'name': 'network-vif-deleted', 'tag': '15ed134d-debb-4ac0-8153-a8ad4f7f255e', 'status': 'completed', 'code': 200} May 31 12:07:42.879847 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-388e05f9-15b0-4a57-8ef3-21e42f27876c req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "DELETE /v2.0/ports/15ed134d-debb-4ac0-8153-a8ad4f7f255e HTTP/1.1" status: 204 len: 173 time: 1.3912921 May 31 12:07:42.926714 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:07:43.234725 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:07:43.238355 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:07:43.247541 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:07:43.248147 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:07:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:07:43.424256 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:07:43.684422 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-316217f2-f175-498d-9e8a-51608a2672c2 req-2562923b-2a52-4413-9900-3722c92de3c0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=58d18350-b3dd-4395-aea6-39d797cb95a1 HTTP/1.1" status: 200 len: 210 time: 0.0474927 May 31 12:07:43.751280 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:07:43.755454 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2c6c3875-9b24-44f1-b38b-83656203a0ce None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:07:43.957055 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f4f4e9ec-9d5f-4b4a-852e-e62b3225626e None None] New status for floating IP e64c4829-1a2c-4e4b-832d-03e4b32f81f4: DOWN {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:07:45.020993 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d567df87-c936-48b4-8723-daf23a073426 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-2001634369&fields=id&tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 267 time: 0.0482614 May 31 12:07:45.198750 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-96d0f8b3-fb39-4bb0-a5c8-8881627445ba tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:45.201256 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-96d0f8b3-fb39-4bb0-a5c8-8881627445ba tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 811 time: 0.1732974 May 31 12:07:45.215719 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-18fb06e9-901c-4ad2-a984-80574aab5f83 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/quotas/0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 375 time: 0.0092509 May 31 12:07:45.300766 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-45b25fb8-76bb-4326-aeb2-9cbecf535cbd tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0802128 May 31 12:07:45.513512 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-809b5fac-27e7-4b12-b6e6-02f000fb5db5 service neutron] 10.222.0.48 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2007439 May 31 12:07:45.644431 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-005636c3-9691-4ffe-ac08-240e54478108 service neutron] 10.222.0.48 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1251681 May 31 12:07:46.259902 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 15ed134d-debb-4ac0-8153-a8ad4f7f255e no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:07:46.296883 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 15ed134d-debb-4ac0-8153-a8ad4f7f255e {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:07:46.297238 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 15ed134d-debb-4ac0-8153-a8ad4f7f255e not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:07:46.339902 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-0cdfe4b2-b3fa-4531-9ef9-b4e498ff56e6 req-52cdca2b-4449-4753-84e9-122b694048e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:07:46.418211 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-0cdfe4b2-b3fa-4531-9ef9-b4e498ff56e6 req-52cdca2b-4449-4753-84e9-122b694048e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:07:46.419373 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0cdfe4b2-b3fa-4531-9ef9-b4e498ff56e6 req-52cdca2b-4449-4753-84e9-122b694048e8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 210 time: 0.1255579 May 31 12:07:47.568152 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-6a7fe68c-403a-4a48-9c5f-7879082b7d2b req-d35fd1ec-6299-4a53-ba85-c39c90db5076 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 210 time: 0.0427785 May 31 12:07:48.329292 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-741800c3-c715-40d5-9587-0c152a7085ef tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:48.330755 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-741800c3-c715-40d5-9587-0c152a7085ef tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 811 time: 0.1470864 May 31 12:07:48.385164 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-7750585b-35a9-4eec-8a89-fca7fedc7b27 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 380 time: 0.0434914 May 31 12:07:48.393153 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'port': {'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'admin_state_up': True, 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'security_groups': ['bd77d026-8f31-4ec6-a2ff-f8e6a6857859']}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:07:48.530074 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:07:48.539942 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:07:48.618108 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Made reservation on behalf of 0e2c6e0e5c334c60a47f2dc2d0c79653 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:07:48.621869 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:48.732775 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-455de259-6d9e-4c36-8362-9782707115b3 req-343d7684-05f7-40ff-b0f8-9dc79c7ae0b4 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 210 time: 0.0552428 May 31 12:07:48.824709 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Allocated IP 172.20.0.3 (8a4efbb8-aa93-49be-a073-41ce66e25d2f/4a96f417-9ce3-448a-9f7f-529c69a0ace7/dc081ad4-14a2-4ac7-8d8e-3b836037dc43) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:07:49.011156 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:49.026453 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Transition to ACTIVE for port object dc081ad4-14a2-4ac7-8d8e-3b836037dc43 will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:07:49.101541 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:49.108004 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e0642932-7ab2-4acc-85c8-d9d103ae8518 tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:07:49.175215 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:07:49.178417 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:07:49.180856 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:07:49.224188 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:07:49.227247 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:07:49.254748 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:49.255917 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:49.361798 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-d23848a0-b279-4970-abbd-262d858c9f51 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.9720697 May 31 12:07:49.373412 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Request body: {'port': {'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:07:49.488651 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:49.524286 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-3d72bc51-ff22-4a58-9b44-1b0d8a40fda0 None None] Provisioning for port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:07:49.529154 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-3d72bc51-ff22-4a58-9b44-1b0d8a40fda0 None None] Provisioning complete for port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:07:49.538241 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3d72bc51-ff22-4a58-9b44-1b0d8a40fda0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:49.585016 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-3d72bc51-ff22-4a58-9b44-1b0d8a40fda0 None None] Port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:07:49.930713 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a596e1ef-6db1-4dde-ad34-974f4acf7581 req-8a42eae0-c81f-41fd-89d9-bee8409eb890 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:49.932154 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a596e1ef-6db1-4dde-ad34-974f4acf7581 req-8a42eae0-c81f-41fd-89d9-bee8409eb890 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 931 time: 0.0932741 May 31 12:07:49.943560 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id dc081ad4-14a2-4ac7-8d8e-3b836037dc43 {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:07:49.951690 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:49.966085 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Transition to ACTIVE for port object dc081ad4-14a2-4ac7-8d8e-3b836037dc43 will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:07:49.976889 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a596e1ef-6db1-4dde-ad34-974f4acf7581 req-124b9716-f20a-48ff-ae9f-81645dac2dec tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0389132 May 31 12:07:50.029543 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:50.033433 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-6b76cbb3-8cea-4e3b-ba9c-3765a3fd43d9 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:07:50.162796 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Pushing event updated for resources: {'Port': ['ID=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:07:50.189826 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:07:50.267864 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Attempting to bind port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:07:50.268187 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Attempting to bind port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:07:50.268504 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Attempting to bind port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:07:50.273650 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 12:07:50.273965 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Attempting to bind port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:07:50.279084 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 7, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:07:50.279409 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Checking segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:07:50.279788 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Bound using segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:07:50.280201 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Bound port: dc081ad4-14a2-4ac7-8d8e-3b836037dc43, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:07:50.280539 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:50.374593 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] For port dc081ad4-14a2-4ac7-8d8e-3b836037dc43, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:07:50.377394 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] For port dc081ad4-14a2-4ac7-8d8e-3b836037dc43, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:07:50.467382 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:50.471101 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b0f5f1cc-3d0f-44c7-8405-a66be0b43868 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:07:50.530051 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:50.530658 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:50.532025 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] 10.222.0.47 "PUT /v2.0/ports/dc081ad4-14a2-4ac7-8d8e-3b836037dc43 HTTP/1.1" status: 200 len: 1093 time: 1.1629250 May 31 12:07:50.532529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:07:50.534786 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-b5e58353-d983-43de-a09e-16c6764836d4" -d '{"events": [{"name": "network-changed", "server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:07:50.552858 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Pushing event updated for resources: {'Port': ['ID=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:07:50.556277 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:07:50.585840 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:07:50 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-0c4244ac-98b1-4f85-89c5-773717db38fb x-openstack-request-id: req-0c4244ac-98b1-4f85-89c5-773717db38fb {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:07:50.591095 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:07:50.591095 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-0c4244ac-98b1-4f85-89c5-773717db38fb {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:07:50.591095 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'status': 'completed', 'code': 200} May 31 12:07:50.684036 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5e58353-d983-43de-a09e-16c6764836d4 req-da3c51b9-b5d4-48e2-91dc-9c3c81a0f6e7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:50.685550 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b5e58353-d983-43de-a09e-16c6764836d4 req-da3c51b9-b5d4-48e2-91dc-9c3c81a0f6e7 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1096 time: 0.0793514 May 31 12:07:50.828207 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5e58353-d983-43de-a09e-16c6764836d4 req-16f8a212-2618-4117-b341-42768be508bd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:50.829530 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b5e58353-d983-43de-a09e-16c6764836d4 req-16f8a212-2618-4117-b341-42768be508bd service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&shared=False HTTP/1.1" status: 200 len: 912 time: 0.1366577 May 31 12:07:50.898477 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f668b644-e5f8-4861-b3f3-4dcda5b5e90e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:50.967357 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5e58353-d983-43de-a09e-16c6764836d4 req-06b17be3-3d33-480a-bc62-74bb129926bf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:50.968776 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-b5e58353-d983-43de-a09e-16c6764836d4 req-06b17be3-3d33-480a-bc62-74bb129926bf service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1334345 May 31 12:07:51.077087 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-1d7ce00e-0d39-436d-8eb5-f8f472bab84c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:51.078735 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-1d7ce00e-0d39-436d-8eb5-f8f472bab84c service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1096 time: 0.0748277 May 31 12:07:51.149508 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-9ba9534d-4bf7-45c5-bc60-94636a529f62 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43 HTTP/1.1" status: 200 len: 217 time: 0.0634100 May 31 12:07:51.176121 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ce0ab49-57bb-4f66-8752-f2368425b86f req-6d0eaffe-a28c-461d-936d-89f001d2dcc4 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:51.179812 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1ce0ab49-57bb-4f66-8752-f2368425b86f req-6d0eaffe-a28c-461d-936d-89f001d2dcc4 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 943 time: 0.0800021 May 31 12:07:51.213635 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-caf6c213-af9b-4bc5-acad-f6729331ae2c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:51.215126 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-caf6c213-af9b-4bc5-acad-f6729331ae2c service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0596781 May 31 12:07:51.229560 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1ce0ab49-57bb-4f66-8752-f2368425b86f req-8b86213f-8218-4769-bc1a-a601a62fc06f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0457754 May 31 12:07:51.300089 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-05d62d72-7dea-4b9b-b1bc-2518dbfda2c9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:51.300089 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-05d62d72-7dea-4b9b-b1bc-2518dbfda2c9 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0727973 May 31 12:07:51.433110 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-0cac7ae1-2746-474c-9fd3-44f079f6568a service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1304705 May 31 12:07:51.564689 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-07f93df1-56ea-4610-a7a6-229d8afb5a47 req-5c0dd066-5bb4-4ef2-93d3-88fba6a58e69 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1251905 May 31 12:07:52.415876 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9139d348-3dc6-4bc6-b46a-05ac433cb149 req-eb677f8b-b983-46ab-9e34-9c335bf2d453 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:52.416564 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-9139d348-3dc6-4bc6-b46a-05ac433cb149 req-eb677f8b-b983-46ab-9e34-9c335bf2d453 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:07:52.512817 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-9139d348-3dc6-4bc6-b46a-05ac433cb149 req-eb677f8b-b983-46ab-9e34-9c335bf2d453 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:07:52.513927 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9139d348-3dc6-4bc6-b46a-05ac433cb149 req-eb677f8b-b983-46ab-9e34-9c335bf2d453 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 943 time: 0.1764641 May 31 12:07:52.558159 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-9139d348-3dc6-4bc6-b46a-05ac433cb149 req-9159adf9-34fc-4712-b64e-7b1dd8b57990 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0386510 May 31 12:07:53.360584 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:53.454984 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f68de03c-41ab-44b6-9317-c98c86e0ebe6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:53.751720 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9da05e74-bdc4-48b4-89c9-28586dd316be req-5d863c20-89e5-4629-9e66-5b7048c9dc2f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:53.753412 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9da05e74-bdc4-48b4-89c9-28586dd316be req-5d863c20-89e5-4629-9e66-5b7048c9dc2f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 943 time: 0.0899856 May 31 12:07:53.796742 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9da05e74-bdc4-48b4-89c9-28586dd316be req-e1ac29d8-76bb-4ce1-96ef-cb8fc9429c46 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0377028 May 31 12:07:54.638125 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:54.981620 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-917d0079-0ef6-4272-a225-3cd91e486008 req-719b7314-20aa-4b4a-b4c2-71635bedcfd7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:54.983027 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-917d0079-0ef6-4272-a225-3cd91e486008 req-719b7314-20aa-4b4a-b4c2-71635bedcfd7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 943 time: 0.0840130 May 31 12:07:55.111119 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-917d0079-0ef6-4272-a225-3cd91e486008 req-65410554-787f-41ef-b7af-4bfdd5e27de7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1224351 May 31 12:07:56.129415 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:07:56.409204 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63a96d66-b007-4207-a233-f5a747ad57e4 req-f728e8b1-b5e8-4636-bfc1-c72926655f7e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:56.410667 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-63a96d66-b007-4207-a233-f5a747ad57e4 req-f728e8b1-b5e8-4636-bfc1-c72926655f7e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 943 time: 0.1885407 May 31 12:07:56.454618 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-63a96d66-b007-4207-a233-f5a747ad57e4 req-6c2aa744-843c-419a-b0b4-87cdb992667c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0377116 May 31 12:07:57.634299 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b8a99a0-42e2-4a76-b315-eba99d378c79 req-3af4d048-954f-48e1-a3f5-2a7e8014d462 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:57.635740 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6b8a99a0-42e2-4a76-b315-eba99d378c79 req-3af4d048-954f-48e1-a3f5-2a7e8014d462 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 943 time: 0.0792534 May 31 12:07:57.683109 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-6b8a99a0-42e2-4a76-b315-eba99d378c79 req-2f351408-318c-496a-b9e7-fbd5c657ead9 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0417747 May 31 12:07:58.865077 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-37a35dff-7177-40c0-a841-4fa5c7259cb7 req-e23c978b-cd83-43a0-a631-78e9b7227259 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:07:58.866441 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-37a35dff-7177-40c0-a841-4fa5c7259cb7 req-e23c978b-cd83-43a0-a631-78e9b7227259 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 943 time: 0.0809364 May 31 12:07:58.911708 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-37a35dff-7177-40c0-a841-4fa5c7259cb7 req-d7e95090-36cf-401d-9a2c-55d4875e01c3 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0390720 May 31 12:08:00.094907 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-be54f0c5-3e41-485b-90be-e22ffdb0bed7 req-8cd885e9-302b-4770-9000-10ec8acf1276 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:00.096331 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-be54f0c5-3e41-485b-90be-e22ffdb0bed7 req-8cd885e9-302b-4770-9000-10ec8acf1276 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 943 time: 0.0791740 May 31 12:08:00.139994 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-be54f0c5-3e41-485b-90be-e22ffdb0bed7 req-8770d815-a27d-4afa-88ce-fa0b1535c778 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0370789 May 31 12:08:00.331869 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device dc081ad4-14a2-4ac7-8d8e-3b836037dc43 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:08:00.462133 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port dc081ad4-14a2-4ac7-8d8e-3b836037dc43, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:08:00.475025 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 362, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:08:00.532477 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:00.548944 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id dc081ad4-14a2-4ac7-8d8e-3b836037dc43 {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:08:00.563640 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port dc081ad4-14a2-4ac7-8d8e-3b836037dc43, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:08:00.640289 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:00.642016 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:08:00.705344 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:08:00.712470 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.712815 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.713150 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.713482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.713844 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.714169 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.714491 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.714814 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.715140 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.715458 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.715776 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.716091 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:08:00.719744 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:08:00.870314 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['dc081ad4-14a2-4ac7-8d8e-3b836037dc43'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:08:01.324807 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ebd9d73-16a3-41fd-86c4-95fbf618d019 req-28208837-9923-40bc-a7ea-098597391472 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:01.326238 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3ebd9d73-16a3-41fd-86c4-95fbf618d019 req-28208837-9923-40bc-a7ea-098597391472 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 944 time: 0.0804327 May 31 12:08:01.369421 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-3ebd9d73-16a3-41fd-86c4-95fbf618d019 req-3f16b604-07b5-4475-89cf-8684d4515dbb tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0375276 May 31 12:08:02.346834 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device dc081ad4-14a2-4ac7-8d8e-3b836037dc43 up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:08:02.468557 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:08:02.474195 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:08:02.474788 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:02.555328 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46b5aa21-789f-4cb1-b34c-9beac8adc868 req-100066a0-43a5-4789-ba03-cac42a28f36e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:02.556780 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-46b5aa21-789f-4cb1-b34c-9beac8adc868 req-100066a0-43a5-4789-ba03-cac42a28f36e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 944 time: 0.0851572 May 31 12:08:02.593521 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:02.600941 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-46b5aa21-789f-4cb1-b34c-9beac8adc868 req-9171796c-4a31-4303-b6f2-5b6b34d22c2d tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0382223 May 31 12:08:02.621299 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:08:02.623521 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-c653df94-d1e2-4937-a7ac-b66370f6171a" -d '{"events": [{"server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "name": "network-vif-plugged", "status": "completed", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:08:02.631903 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port dc081ad4-14a2-4ac7-8d8e-3b836037dc43, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:08:02.706324 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:02.708376 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:08:02.774444 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:08:02.777450 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:08:03.777904 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:08:02 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-f2218551-ce6b-4bf0-bfce-21536bd77398 x-openstack-request-id: req-f2218551-ce6b-4bf0-bfce-21536bd77398 {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:08:03.778531 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "name": "network-vif-plugged", "status": "completed", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:08:03.778644 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-f2218551-ce6b-4bf0-bfce-21536bd77398 {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:08:03.778972 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'code': 200} May 31 12:08:03.783223 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a882e732-c58b-49bf-9723-c682361e53d3 req-4cfe05d1-34a2-4116-a429-5a54b99ddbb7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:03.784672 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a882e732-c58b-49bf-9723-c682361e53d3 req-4cfe05d1-34a2-4116-a429-5a54b99ddbb7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 945 time: 0.0793524 May 31 12:08:03.828796 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-a882e732-c58b-49bf-9723-c682361e53d3 req-11dc6838-a7d8-493c-bdd7-0bfa9e372f0c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0378323 May 31 12:08:04.188519 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-334ab8f1-2ec7-44b6-bc07-7f851df8af02 req-256a3745-57ee-447a-8f36-26165a7f4583 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:04.190020 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-334ab8f1-2ec7-44b6-bc07-7f851df8af02 req-256a3745-57ee-447a-8f36-26165a7f4583 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1098 time: 0.0734582 May 31 12:08:04.231309 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-334ab8f1-2ec7-44b6-bc07-7f851df8af02 req-a8b21897-d84c-403b-b117-177c7d262adb service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0341163 May 31 12:08:05.015461 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4bcb96ce-cb34-46a6-b08c-640865cbf450 req-69b69d4f-62c2-4d58-9c63-e53764baadf1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:05.016911 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4bcb96ce-cb34-46a6-b08c-640865cbf450 req-69b69d4f-62c2-4d58-9c63-e53764baadf1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 945 time: 0.0795081 May 31 12:08:05.065647 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4bcb96ce-cb34-46a6-b08c-640865cbf450 req-d39a02d5-59ea-4cf4-a3a3-8d066c820e15 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0423105 May 31 12:08:06.251427 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18ab2570-7521-41ba-82a0-80920f6252e0 req-6f66d16a-f8f7-4ce2-a68e-f39c7adb6c59 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:06.253098 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-18ab2570-7521-41ba-82a0-80920f6252e0 req-6f66d16a-f8f7-4ce2-a68e-f39c7adb6c59 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 945 time: 0.0803638 May 31 12:08:06.300729 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-18ab2570-7521-41ba-82a0-80920f6252e0 req-5751d5cc-b497-45c2-8202-b4d852bdf3c7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0418067 May 31 12:08:06.483602 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bff7835-7211-4ed4-bb2b-43e02dee2dad req-2bbc288a-c033-44a5-b9ab-9e878760d310 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:06.485032 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8bff7835-7211-4ed4-bb2b-43e02dee2dad req-2bbc288a-c033-44a5-b9ab-9e878760d310 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 945 time: 0.0805144 May 31 12:08:06.531941 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8bff7835-7211-4ed4-bb2b-43e02dee2dad req-ae717d76-6cbe-402c-a441-c41082017364 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0399494 May 31 12:08:06.555457 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40784) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:08:06.626036 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-38d6967c-d4da-4ff2-b1a6-c301a1f1e097 tempest-TestVolumeBootPattern-1447201260 tempest-TestVolumeBootPattern-1447201260-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:06.627717 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-38d6967c-d4da-4ff2-b1a6-c301a1f1e097 tempest-TestVolumeBootPattern-1447201260 tempest-TestVolumeBootPattern-1447201260-project-admin] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1093 time: 0.0710645 May 31 12:08:06.630573 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40788) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:08:06.635393 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address': '172.20.0.3'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:08:06.641037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:08:06.644756 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:floatingip and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:08:06.648341 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for floatingip was recalculated. Used quota:1. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:08:06.654009 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource floatingip. Total usage: 1; quota limit: 50; headroom:49 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:08:06.777079 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:06.917993 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:07.049979 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Allocated IP 172.27.1.78 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/1eb0c86e-5c25-436c-bf8e-b066b71f4d19) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:08:07.190650 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:07.291253 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:07.294790 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-1bce42f1-69d8-4e9d-af3b-8f6847a8ff8e tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:08:07.356026 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:08:07.358779 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:08:07.362141 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:08:07.395910 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=1eb0c86e-5c25-436c-bf8e-b066b71f4d19,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:08:07.399845 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:08:07.766167 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:07.978931 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:08.185617 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:08.245324 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:08.249901 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-eab51e43-dfbf-4b2a-b363-f98d6859ceb2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:08:08.359003 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=1eb0c86e-5c25-436c-bf8e-b066b71f4d19,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:08:08.363075 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:08:08.369870 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:08.547917 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Floating IP 0cea0cc6-fc31-47d9-a884-be7ea8ab0e58 associated. External IP: 172.27.1.78, port: dc081ad4-14a2-4ac7-8d8e-3b836037dc43. May 31 12:08:08.893648 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:08:09.069756 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:08:09.078855 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:08:09.088126 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:08:09.088746 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:07:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:08:09.154374 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:09.209050 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-5cdb7a34-3f8f-4d71-b2a5-c04c8c698580 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 2.5774789 May 31 12:08:09.209896 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:08:09.212140 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-8bb002cb-2589-4f96-bb37-e16348f4ef29" -d '{"events": [{"name": "network-changed", "server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:08:09.254549 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:08:09 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-09d8756d-5aa0-44c0-aa1a-3f823c6bd8a3 x-openstack-request-id: req-09d8756d-5aa0-44c0-aa1a-3f823c6bd8a3 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:08:09.254879 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:08:09.255170 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-09d8756d-5aa0-44c0-aa1a-3f823c6bd8a3 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:08:09.255600 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'status': 'completed', 'code': 200} May 31 12:08:09.270470 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:08:09.352010 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-9818244f-9189-4e4c-bd07-e8e3ad336f92 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:09.353584 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-9818244f-9189-4e4c-bd07-e8e3ad336f92 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1098 time: 0.0759165 May 31 12:08:09.409246 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:09.600582 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-1e7bf2c7-2406-43e4-b945-5b2d181f3226 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:09.602042 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-1e7bf2c7-2406-43e4-b945-5b2d181f3226 service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.2425032 May 31 12:08:09.637379 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:07Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:08:09.641177 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:08:09.649520 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:08:09.650135 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:07:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:08:09.706098 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-fde9c823-285a-48e4-96cc-54f1faacaf7c service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43 HTTP/1.1" status: 200 len: 998 time: 0.0974743 May 31 12:08:09.769130 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-2448a34c-2bd9-438a-b24d-16b71f59a1ca service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:09.770633 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-2448a34c-2bd9-438a-b24d-16b71f59a1ca service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0586658 May 31 12:08:09.850419 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-c914cc91-12f0-4aad-9dda-e36cbc34f758 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:09.851876 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-c914cc91-12f0-4aad-9dda-e36cbc34f758 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0750816 May 31 12:08:09.865782 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:08:09.984751 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-f87e2480-6b3a-47d1-b8b8-ba4b3d380b57 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1256678 May 31 12:08:10.122371 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8bb002cb-2589-4f96-bb37-e16348f4ef29 req-52c8ba0b-4edb-4739-ae1c-8606ea0688c3 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1314247 May 31 12:08:10.213261 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:07Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:08:10.217167 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c0db46d3-21b9-4d83-8dc4-8075710d4319 None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:07Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:08:10.398704 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a4b2db78-8830-46b4-ac14-64ad2ebf190c None None] New status for floating IP 0cea0cc6-fc31-47d9-a884-be7ea8ab0e58: ACTIVE {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:08:13.460866 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:08:20.930461 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9b5b13cc-a403-4fc6-9459-8c4461430b3c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:23.359265 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:23.442036 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3af48659-0ff3-4fbe-8568-c0b0ebebfc36 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:24.651274 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:26.193062 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:39.406880 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:48.028381 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-5daa70df-565a-440e-b475-8c276963fdf8 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0772798 May 31 12:08:48.178271 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-e4c8e1c6-fa58-4e49-9830-bc295d825a4d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:48.179699 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-e4c8e1c6-fa58-4e49-9830-bc295d825a4d service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1098 time: 0.0784397 May 31 12:08:48.322053 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-719820c4-4a6b-4833-8ca5-2be6a6c2ccdd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:48.323389 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-719820c4-4a6b-4833-8ca5-2be6a6c2ccdd service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.1245439 May 31 12:08:48.425328 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-724036ba-f75c-488c-b830-141ba4667f08 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43 HTTP/1.1" status: 200 len: 1000 time: 0.0954709 May 31 12:08:48.490959 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-5ac130f4-79b5-4802-8496-b712afe00a94 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:48.492340 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-5ac130f4-79b5-4802-8496-b712afe00a94 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0604892 May 31 12:08:48.571117 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-e2de9415-4e8b-4025-a703-917ee2c81d24 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:08:48.572654 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-e2de9415-4e8b-4025-a703-917ee2c81d24 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0736623 May 31 12:08:48.704586 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-cf6e9eef-39f1-49cf-9307-44cff0b2acd8 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1241808 May 31 12:08:48.836856 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-88abf5ff-bb61-4caa-8072-f36283f47df1 req-f5b000fc-bd3d-45e9-97b4-6a93a54263ae service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1248143 May 31 12:08:50.459063 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:08:50.903121 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e746189e-bbef-48b7-97a4-719d2075f521 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:53.346711 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:53.432138 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-7d1ceffc-2f4f-4342-8313-7b2f515da395 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:54.638138 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:08:56.163384 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:09.414371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:20.899442 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-81a2aaf8-2c11-4054-9a34-d36c4958f924 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:23.353063 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:23.449874 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-12093391-a77d-403d-828b-56fb18aecd2d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:24.643871 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:26.147181 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:27.458404 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:09:39.397537 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:45.444400 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-967c1221-5bfc-47d1-b029-68f14023d925 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-2001634369&fields=id&tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 267 time: 0.0384748 May 31 12:09:45.596388 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-fc88e0f1-639f-4c9d-be8e-99fe8a8f9cbb tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:45.597797 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-fc88e0f1-639f-4c9d-be8e-99fe8a8f9cbb tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 811 time: 0.1471124 May 31 12:09:45.660113 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-84c27620-87f2-4245-a12b-1c6a83d94207 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/quotas/0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 375 time: 0.0568607 May 31 12:09:45.749192 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-cedb5d61-f2f6-47d6-ab30-62943064bf77 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&fields=id HTTP/1.1" status: 200 len: 440 time: 0.0836172 May 31 12:09:45.884203 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-6e1080f4-33da-4008-8186-1a693e588d51 service neutron] 10.222.0.48 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1236846 May 31 12:09:46.011689 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-78366d58-da7d-4bd1-bd2b-867ae1efce43 service neutron] 10.222.0.48 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1212785 May 31 12:09:46.697000 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ce355e49-5755-4ef8-872a-4d8b6e3d3c43 req-da4b454f-ae86-4fd7-8dbe-8ce12de4207e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 210 time: 0.0953083 May 31 12:09:47.847933 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4bd58590-ffd0-400a-95f4-fe5271632bdb req-6838c0ed-04ac-4266-8a77-1447af52fb8c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 210 time: 0.0444469 May 31 12:09:48.958692 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-64e66be2-8f8c-4a19-95d4-f9acdf27efef tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:48.960148 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-64e66be2-8f8c-4a19-95d4-f9acdf27efef tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 811 time: 0.1498692 May 31 12:09:49.109317 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1769d4eb-f018-4db0-a903-7e221f18189f req-7fa1c685-25b7-4be3-8002-1b90b67be065 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1267757 May 31 12:09:49.127051 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-0d900fbc-40b1-4e6a-be2a-d95829518bc8 req-5d03a742-f32a-48e6-a9fa-3e2404003e53 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 210 time: 0.1598876 May 31 12:09:49.152128 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-b1d1dddf-efc2-450d-a62a-1bc77ad5e96e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653 HTTP/1.1" status: 200 len: 380 time: 0.1833131 May 31 12:09:49.160482 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'port': {'device_id': '0d094ad5-4278-4dce-ac78-413a587effbc', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'admin_state_up': True, 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'security_groups': ['bd77d026-8f31-4ec6-a2ff-f8e6a6857859']}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:09:49.267830 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1769d4eb-f018-4db0-a903-7e221f18189f req-6d40f810-8497-46ac-b61f-8cc4395bec4f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:49.269300 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1769d4eb-f018-4db0-a903-7e221f18189f req-6d40f810-8497-46ac-b61f-8cc4395bec4f service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1098 time: 0.0739691 May 31 12:09:49.299272 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:09:49.309526 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource port. Total usage: 4; quota limit: 500; headroom:496 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:09:49.407290 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Made reservation on behalf of 0e2c6e0e5c334c60a47f2dc2d0c79653 for: {'port': 1} {{(pid=99040) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 31 12:09:49.410783 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:49.415705 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1769d4eb-f018-4db0-a903-7e221f18189f req-20b0d192-4e26-4872-baac-2baa756366ac service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:49.417156 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1769d4eb-f018-4db0-a903-7e221f18189f req-20b0d192-4e26-4872-baac-2baa756366ac service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.1289330 May 31 12:09:49.528365 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1769d4eb-f018-4db0-a903-7e221f18189f req-da48d1ae-65b2-4a23-9bac-ec2a1ebeea19 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43 HTTP/1.1" status: 200 len: 1000 time: 0.1049058 May 31 12:09:49.582955 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Allocated IP 172.20.0.8 (8a4efbb8-aa93-49be-a073-41ce66e25d2f/4a96f417-9ce3-448a-9f7f-529c69a0ace7/53f9e6bd-0116-4e65-95a5-b197d29610cf) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:09:49.592407 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1769d4eb-f018-4db0-a903-7e221f18189f req-2b57615b-6a82-4702-a2da-960b5a2e82d2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:49.593965 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1769d4eb-f018-4db0-a903-7e221f18189f req-2b57615b-6a82-4702-a2da-960b5a2e82d2 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0593226 May 31 12:09:49.673513 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1769d4eb-f018-4db0-a903-7e221f18189f req-cbfceac4-ff78-4d01-b0c3-61fca0511740 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:49.675008 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1769d4eb-f018-4db0-a903-7e221f18189f req-cbfceac4-ff78-4d01-b0c3-61fca0511740 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0751677 May 31 12:09:49.774585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:49.790857 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Transition to ACTIVE for port object 53f9e6bd-0116-4e65-95a5-b197d29610cf will not be triggered until provisioned by entity DHCP. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:09:49.809554 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1769d4eb-f018-4db0-a903-7e221f18189f req-186052c4-0b56-4155-843a-dd99a6960014 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1275640 May 31 12:09:49.895008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:49.901086 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-27f2b6fa-b21e-4841-9076-f528dbd0ac4e tempest-VolumesActionsTest-968700424 tempest-VolumesActionsTest-968700424-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:09:49.974901 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:09:49.978202 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:09:49.981998 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:09:50.013189 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=53f9e6bd-0116-4e65-95a5-b197d29610cf,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:09:50.027674 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.028056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.028422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.028794 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.029272 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.030176 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.030586 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.030892 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.031248 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.031965 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.031965 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.032297 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:09:50.041455 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.140s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:09:50.054045 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1769d4eb-f018-4db0-a903-7e221f18189f req-13c44c85-1732-4ecc-8566-3c065309f9e6 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2379289 May 31 12:09:50.064687 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:50.065322 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:50.163587 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-669f8291-2e7c-4f62-b56d-03cc1a6359e7 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.0067081 May 31 12:09:50.175524 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Request body: {'port': {'device_id': '0d094ad5-4278-4dce-ac78-413a587effbc', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:09:50.242701 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-4eb47063-b40b-49ec-95c6-c9f088ee91d3 None None] Provisioning for port 53f9e6bd-0116-4e65-95a5-b197d29610cf completed by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:09:50.248614 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-4eb47063-b40b-49ec-95c6-c9f088ee91d3 None None] Provisioning complete for port 53f9e6bd-0116-4e65-95a5-b197d29610cf triggered by entity DHCP. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:09:50.249158 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-4eb47063-b40b-49ec-95c6-c9f088ee91d3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:50.299456 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:50.306468 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-4eb47063-b40b-49ec-95c6-c9f088ee91d3 None None] Port 53f9e6bd-0116-4e65-95a5-b197d29610cf cannot update to ACTIVE because it is not bound. {{(pid=99041) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 31 12:09:50.319512 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d872269-ab46-4f5f-8e08-355e758d9982 req-6a4c9b13-829c-46a9-a392-b1a1afdacb24 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:50.321289 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9d872269-ab46-4f5f-8e08-355e758d9982 req-6a4c9b13-829c-46a9-a392-b1a1afdacb24 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 931 time: 0.0859182 May 31 12:09:50.371374 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-9d872269-ab46-4f5f-8e08-355e758d9982 req-033746b6-b709-49eb-9da5-19df67b5b466 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0423901 May 31 12:09:50.643529 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 53f9e6bd-0116-4e65-95a5-b197d29610cf {{(pid=99040) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:09:50.650496 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:50.663978 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.provisioning_blocks [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Transition to ACTIVE for port object 53f9e6bd-0116-4e65-95a5-b197d29610cf will not be triggered until provisioned by entity L2. {{(pid=99040) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 31 12:09:50.716427 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:50.720039 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9c192fe5-6a41-4d4d-a5ef-f2cfbf820ad0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:09:50.879482 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Pushing event updated for resources: {'Port': ['ID=53f9e6bd-0116-4e65-95a5-b197d29610cf,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:09:50.889889 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:09:50.911166 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-83b7ae6a-a66d-41a5-a198-91a37e493aaf None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:50.964916 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Attempting to bind port 53f9e6bd-0116-4e65-95a5-b197d29610cf on host hv-cinder-79123 for vnic_type normal with profile {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 31 12:09:50.965241 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Attempting to bind port 53f9e6bd-0116-4e65-95a5-b197d29610cf by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 31 12:09:50.965559 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Attempting to bind port 53f9e6bd-0116-4e65-95a5-b197d29610cf on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:09:50.970690 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Port 53f9e6bd-0116-4e65-95a5-b197d29610cf on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 31 12:09:50.971001 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Attempting to bind port 53f9e6bd-0116-4e65-95a5-b197d29610cf on network 8a4efbb8-aa93-49be-a073-41ce66e25d2f {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 31 12:09:50.975950 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Checking agent: {'id': '6ba78f2e-fa11-4ad0-b381-2745f79dacfb', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'started_at': datetime.datetime(2021, 5, 31, 11, 52, 39), 'heartbeat_timestamp': datetime.datetime(2021, 5, 31, 12, 9, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 31 12:09:50.976548 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Checking segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99040) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 31 12:09:50.976664 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Bound using segment: {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'} {{(pid=99040) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 31 12:09:50.977031 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.managers [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Bound port: 53f9e6bd-0116-4e65-95a5-b197d29610cf, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'a29d566a-c32d-4472-972d-990571fdae77', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 362, 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f'}}] {{(pid=99040) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 31 12:09:50.977366 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:51.068472 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] For port 53f9e6bd-0116-4e65-95a5-b197d29610cf, host hv-cinder-79123, cleared binding levels {{(pid=99040) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 31 12:09:51.071359 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] For port 53f9e6bd-0116-4e65-95a5-b197d29610cf, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=53f9e6bd-0116-4e65-95a5-b197d29610cf,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99040) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 31 12:09:51.157827 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:51.161765 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0daa39a7-fdaa-4809-b141-2b20e29b40eb tempest-VolumesSnapshotTestJSON-353941298 tempest-VolumesSnapshotTestJSON-353941298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:09:51.219138 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:51.219735 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:51.221063 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] 10.222.0.47 "PUT /v2.0/ports/53f9e6bd-0116-4e65-95a5-b197d29610cf HTTP/1.1" status: 200 len: 1093 time: 1.0503573 May 31 12:09:51.221611 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:09:51.223833 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-ed72cada-5060-47ca-81aa-4c1fd5bbdb77" -d '{"events": [{"name": "network-changed", "server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:09:51.243746 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Pushing event updated for resources: {'Port': ['ID=53f9e6bd-0116-4e65-95a5-b197d29610cf,revision_number=3']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:09:51.247056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-3d85448f-c40a-4b86-9248-5f2162b4e5be service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:09:51.270857 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:09:51 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-2784fd43-2dc1-4c23-8328-f71377303939 x-openstack-request-id: req-2784fd43-2dc1-4c23-8328-f71377303939 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:09:51.270857 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:09:51.270857 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-2784fd43-2dc1-4c23-8328-f71377303939 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:09:51.270857 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'status': 'completed', 'code': 200} May 31 12:09:51.362326 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ed72cada-5060-47ca-81aa-4c1fd5bbdb77 req-72388060-1675-4ab0-807b-1de6d95ce547 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:51.363835 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ed72cada-5060-47ca-81aa-4c1fd5bbdb77 req-72388060-1675-4ab0-807b-1de6d95ce547 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 1096 time: 0.0740454 May 31 12:09:51.575100 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34eeb96a-097a-4bb4-9f7e-4f37c7df855b req-100bb9c2-88c0-4ff5-bb4a-c5ec924b2423 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:51.577096 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-34eeb96a-097a-4bb4-9f7e-4f37c7df855b req-100bb9c2-88c0-4ff5-bb4a-c5ec924b2423 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 943 time: 0.0900695 May 31 12:09:51.601919 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ed72cada-5060-47ca-81aa-4c1fd5bbdb77 req-b97e52e6-427d-47fb-a390-d53c39864a3e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:51.603259 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ed72cada-5060-47ca-81aa-4c1fd5bbdb77 req-b97e52e6-427d-47fb-a390-d53c39864a3e service neutron] 10.222.0.47 "GET /v2.0/networks?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&shared=False HTTP/1.1" status: 200 len: 912 time: 0.2323682 May 31 12:09:51.620964 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-34eeb96a-097a-4bb4-9f7e-4f37c7df855b req-b8d6b5f5-118c-477f-8561-c411cc86f51e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0382061 May 31 12:09:51.738033 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ed72cada-5060-47ca-81aa-4c1fd5bbdb77 req-99bd4e57-0829-4686-be78-5b665cbbd274 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:51.739349 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ed72cada-5060-47ca-81aa-4c1fd5bbdb77 req-99bd4e57-0829-4686-be78-5b665cbbd274 service neutron] 10.222.0.47 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1298137 May 31 12:09:51.844723 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-9eca44c8-1ae6-4c1e-9898-3afd0a75a541 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:51.846194 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-9eca44c8-1ae6-4c1e-9898-3afd0a75a541 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 1096 time: 0.0689952 May 31 12:09:51.916061 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-f3bdfba7-82a9-4109-a265-943b6f235612 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=53f9e6bd-0116-4e65-95a5-b197d29610cf HTTP/1.1" status: 200 len: 217 time: 0.0627685 May 31 12:09:51.976020 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-1096bfc3-597b-41c9-973e-ca62a39c4a76 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:51.977403 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-1096bfc3-597b-41c9-973e-ca62a39c4a76 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0559552 May 31 12:09:52.050704 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-4482a2cb-398b-416d-a589-edef9bf70a43 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:52.052107 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-4482a2cb-398b-416d-a589-edef9bf70a43 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0680711 May 31 12:09:52.182067 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-fcf1e10a-6519-47ed-8410-cb19b4db6f40 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1220944 May 31 12:09:52.311227 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-8649d25b-1fed-4cc7-af80-6414a85de41e req-f3560d94-ee69-4a4b-a68b-f3340faa3e62 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1221275 May 31 12:09:52.820211 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0a029b5-d637-46f6-898e-ee3630ebdd45 req-56bfe5a6-dfb8-43a7-9450-719792e5b3a0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:52.820852 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c0a029b5-d637-46f6-898e-ee3630ebdd45 req-56bfe5a6-dfb8-43a7-9450-719792e5b3a0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:09:52.893330 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-c0a029b5-d637-46f6-898e-ee3630ebdd45 req-56bfe5a6-dfb8-43a7-9450-719792e5b3a0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:09:52.894484 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0a029b5-d637-46f6-898e-ee3630ebdd45 req-56bfe5a6-dfb8-43a7-9450-719792e5b3a0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 943 time: 0.1677613 May 31 12:09:52.939302 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0a029b5-d637-46f6-898e-ee3630ebdd45 req-69b65995-5182-4e46-8883-e0146d5130b9 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391300 May 31 12:09:53.379155 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:53.436923 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-d1369c5c-e72c-4266-a771-f397c762d833 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:54.123390 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4085cf88-9d0e-4ad2-a546-07f560c26913 req-e5b9e977-3bcd-4aec-8398-43329b5cd482 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:54.124808 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4085cf88-9d0e-4ad2-a546-07f560c26913 req-e5b9e977-3bcd-4aec-8398-43329b5cd482 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 943 time: 0.0830240 May 31 12:09:54.172361 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-4085cf88-9d0e-4ad2-a546-07f560c26913 req-bc7f2e08-14d2-4bb2-be09-1774c79f9bc2 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0419431 May 31 12:09:54.682510 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:55.360972 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75afa03e-12a0-42f2-85bd-85f5b6752726 req-04511834-37b9-448a-acfa-7866187923bb tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:55.362423 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-75afa03e-12a0-42f2-85bd-85f5b6752726 req-04511834-37b9-448a-acfa-7866187923bb tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 943 time: 0.0858705 May 31 12:09:55.405591 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-75afa03e-12a0-42f2-85bd-85f5b6752726 req-7d29bbf4-7abb-4d5b-b1be-a69a52a65ce1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0376136 May 31 12:09:55.676622 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [req-f21012f6-9cfa-442f-b326-42af02da004c req-b3ab07c8-afb3-4f12-87c9-d4470c04f48f service neutron] (99039) timed out ('10.222.0.47', 49565) {{(pid=99039) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 31 12:09:56.167829 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:09:56.586761 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-abaa3872-446a-4f57-8059-37edb9904b75 req-95e5adca-4378-4434-9f5c-bc27024bee26 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:56.588173 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-abaa3872-446a-4f57-8059-37edb9904b75 req-95e5adca-4378-4434-9f5c-bc27024bee26 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 943 time: 0.0802572 May 31 12:09:56.633160 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-abaa3872-446a-4f57-8059-37edb9904b75 req-a81780f8-f8c0-473b-adca-66905b409d28 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0392153 May 31 12:09:57.817356 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee8a222a-9b0f-4b2d-9346-650c234186a4 req-f3a261b8-ed4e-4c1e-8091-c1ff03cdc0f2 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:57.818796 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee8a222a-9b0f-4b2d-9346-650c234186a4 req-f3a261b8-ed4e-4c1e-8091-c1ff03cdc0f2 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 943 time: 0.0841484 May 31 12:09:57.862823 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-ee8a222a-9b0f-4b2d-9346-650c234186a4 req-e62d4d92-1bd7-42e5-9002-ba7c6414c101 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0385430 May 31 12:09:59.049086 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d00c3142-8c58-4173-8c0a-5e15012ae37f req-62468f23-4011-44df-83cf-a54fb55a47ea tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:09:59.050550 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d00c3142-8c58-4173-8c0a-5e15012ae37f req-62468f23-4011-44df-83cf-a54fb55a47ea tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 943 time: 0.0828307 May 31 12:09:59.095158 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-d00c3142-8c58-4173-8c0a-5e15012ae37f req-3b4c07c9-b55a-4801-b0a5-fc84fe0d470f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0387182 May 31 12:09:59.991831 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [req-ee949f14-d1c8-4b05-a476-55316c7a556b req-e707c5ea-800d-4563-ae3d-0ce57ed9d30a service neutron] (99040) timed out ('10.222.0.47', 49549) {{(pid=99040) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 31 12:10:00.278386 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-591864b0-6e70-4e17-90bb-99982e013202 req-fc92dbd8-9fc3-4ca5-b9b4-613ce258849f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:00.280351 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-591864b0-6e70-4e17-90bb-99982e013202 req-fc92dbd8-9fc3-4ca5-b9b4-613ce258849f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 943 time: 0.0831454 May 31 12:10:00.324293 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-591864b0-6e70-4e17-90bb-99982e013202 req-7aa409f0-0a11-45aa-a834-9efd3ee7ab5a tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0380757 May 31 12:10:01.014237 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 53f9e6bd-0116-4e65-95a5-b197d29610cf details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=99041) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 31 12:10:01.157501 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 53f9e6bd-0116-4e65-95a5-b197d29610cf, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=53f9e6bd-0116-4e65-95a5-b197d29610cf,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:10:01.170725 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Returning: {'device': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'port_id': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'mac_address': 'fa:16:3e:f2:ac:66', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 362, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99041) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 31 12:10:01.224371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:01.243789 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 53f9e6bd-0116-4e65-95a5-b197d29610cf {{(pid=99041) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 31 12:10:01.259138 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 53f9e6bd-0116-4e65-95a5-b197d29610cf, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=53f9e6bd-0116-4e65-95a5-b197d29610cf,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:10:01.314814 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:01.316462 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:10:01.381925 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=53f9e6bd-0116-4e65-95a5-b197d29610cf,revision_number=4']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:10:01.389120 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.389458 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.389823 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.390155 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.390482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.390811 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.391138 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.391456 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.391777 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.392094 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.392409 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.392726 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:10:01.396468 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:10:01.510884 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c350694e-5bb5-4a4e-9383-da0278bf8186 req-daec7a4b-95fd-4ab5-990b-ac54e89fe851 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:01.512318 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c350694e-5bb5-4a4e-9383-da0278bf8186 req-daec7a4b-95fd-4ab5-990b-ac54e89fe851 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 944 time: 0.0807455 May 31 12:10:01.534371 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] get_ports_and_sgs() called for port_ids ['53f9e6bd-0116-4e65-95a5-b197d29610cf'] {{(pid=99041) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 31 12:10:01.558826 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-c350694e-5bb5-4a4e-9383-da0278bf8186 req-ca2cbafe-e5ab-4866-a5f0-d9d939c90ead tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0406275 May 31 12:10:02.742285 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cf44e4a-8085-4c16-a25b-0bd070695ce5 req-9a20abc8-f5fc-430d-94f7-971cfeb9fae9 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:02.743708 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1cf44e4a-8085-4c16-a25b-0bd070695ce5 req-9a20abc8-f5fc-430d-94f7-971cfeb9fae9 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 944 time: 0.0798526 May 31 12:10:02.787844 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-1cf44e4a-8085-4c16-a25b-0bd070695ce5 req-71ea7748-eade-488a-af3f-c048b76d978d tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0379083 May 31 12:10:03.030954 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 53f9e6bd-0116-4e65-95a5-b197d29610cf up at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 31 12:10:03.160945 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning for port 53f9e6bd-0116-4e65-95a5-b197d29610cf completed by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 31 12:10:03.165639 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.provisioning_blocks [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Provisioning complete for port 53f9e6bd-0116-4e65-95a5-b197d29610cf triggered by entity L2. {{(pid=99041) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 31 12:10:03.166036 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1545003'] for port, provisioning_complete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:03.280476 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:03.306326 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf'}] {{(pid=99041) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:10:03.308506 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}2f8768fcc6877eb95ea0f642cdd59be30b5f4dbadf2153d7f88f84081d5480eb" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-3fe8d826-c434-453d-8b31-a7ef850335dd" -d '{"events": [{"server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "name": "network-vif-plugged", "status": "completed", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf"}]}' {{(pid=99041) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:10:03.316361 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] For port 53f9e6bd-0116-4e65-95a5-b197d29610cf, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=53f9e6bd-0116-4e65-95a5-b197d29610cf,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:10:03.350698 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:10:03 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-945663f2-2527-4367-ae30-01f52169128a x-openstack-request-id: req-945663f2-2527-4367-ae30-01f52169128a {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:10:03.351057 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "name": "network-vif-plugged", "status": "completed", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf", "code": 200}]} {{(pid=99041) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:10:03.351322 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-945663f2-2527-4367-ae30-01f52169128a {{(pid=99041) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:10:03.351715 dv-cinder-791239-4-iscsi neutron-server[99041]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'code': 200} May 31 12:10:03.384770 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790'] for port, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:03.386639 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:10:03.452566 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Pushing event updated for resources: {'Port': ['ID=53f9e6bd-0116-4e65-95a5-b197d29610cf,revision_number=5']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:10:03.455540 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:10:03.767255 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-54b4ad80-61c4-48b6-9d03-c2e5fd8ac319 req-24cc2ae1-9265-4371-a295-be79e2348323 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:03.768613 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-54b4ad80-61c4-48b6-9d03-c2e5fd8ac319 req-24cc2ae1-9265-4371-a295-be79e2348323 service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 1098 time: 0.0730276 May 31 12:10:03.808579 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-54b4ad80-61c4-48b6-9d03-c2e5fd8ac319 req-756206de-064e-4594-84b3-9f30bfd59393 service neutron] 10.222.0.47 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0340884 May 31 12:10:03.966261 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c413c920-4269-43a5-87f5-da1bebc466c4 req-2de5b624-93dd-41f7-9c8e-5092525c90b1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:03.968166 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c413c920-4269-43a5-87f5-da1bebc466c4 req-2de5b624-93dd-41f7-9c8e-5092525c90b1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 945 time: 0.0789652 May 31 12:10:04.014894 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c413c920-4269-43a5-87f5-da1bebc466c4 req-bc472a33-0e41-4907-a8e9-e844813cb719 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0411451 May 31 12:10:04.457443 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:10:05.249799 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-265ac0fa-d4d8-4191-89d2-ac8706fae0b1 req-588e8fc8-3cf6-4c1e-9037-7d302ee261d3 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:05.251222 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-265ac0fa-d4d8-4191-89d2-ac8706fae0b1 req-588e8fc8-3cf6-4c1e-9037-7d302ee261d3 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 945 time: 0.0836740 May 31 12:10:05.293922 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-265ac0fa-d4d8-4191-89d2-ac8706fae0b1 req-45281b3a-e98d-43d7-967e-e3d0cb4009a3 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0371823 May 31 12:10:05.473939 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41929a9a-030e-40d2-857a-f820621fd070 req-0d01950e-83a2-49ce-86b9-5a0e444cf341 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:05.475357 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-41929a9a-030e-40d2-857a-f820621fd070 req-0d01950e-83a2-49ce-86b9-5a0e444cf341 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 945 time: 0.0788879 May 31 12:10:05.519889 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-41929a9a-030e-40d2-857a-f820621fd070 req-b5ccf75a-cabb-4a2d-9ae2-2cf2b07970af tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0390189 May 31 12:10:05.542811 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40922) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:10:05.614390 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-24143dd0-143f-4470-ab97-a2656673ffed tempest-TestVolumeBootPattern-1447201260 tempest-TestVolumeBootPattern-1447201260-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:05.615729 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-24143dd0-143f-4470-ab97-a2656673ffed tempest-TestVolumeBootPattern-1447201260 tempest-TestVolumeBootPattern-1447201260-project-admin] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 1093 time: 0.0719936 May 31 12:10:05.618507 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40926) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:10:05.623645 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.v2.base [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Request body: {'floatingip': {'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'port_id': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address': '172.20.0.8'}} {{(pid=99040) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 31 12:10:05.629247 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 31 12:10:05.632972 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Usage tracker for resource:floatingip and tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 is out of sync, need to count used quota {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 31 12:10:05.635763 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Quota usage for floatingip was recalculated. Used quota:2. {{(pid=99040) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 31 12:10:05.641932 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.quota.driver [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attempting to reserve 1 items for resource floatingip. Total usage: 2; quota limit: 50; headroom:48 {{(pid=99040) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 31 12:10:05.698696 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:05.915561 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:06.046727 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_common [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Allocated IP 172.27.1.99 (8a73d753-6759-4d77-a99f-016e97b0a680/4b97143c-47ad-4e8e-b97f-9970a81c7a23/0ebe55f9-2a7a-4aba-87b1-b26f07199cdc) {{(pid=99040) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 31 12:10:06.199390 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:06.303990 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8759696991699', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:06.307755 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-e392fb34-f0ce-4a56-a746-203d50b1a228 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:10:06.370855 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Network 8a73d753-6759-4d77-a99f-016e97b0a680 is already hosted by enough agents. {{(pid=99040) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 31 12:10:06.374550 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.base_scheduler [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Resources bound (agent type, host, resource id): {{(pid=99040) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 31 12:10:06.378045 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Received port after_create {{(pid=99040) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 31 12:10:06.406737 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=0ebe55f9-2a7a-4aba-87b1-b26f07199cdc,revision_number=1']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:10:06.412497 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:10:06.786433 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:06.916990 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1458973'] for port, before_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:07.117531 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for port, precommit_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:07.206269 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8759697376404', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8759696991717', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8759696992790', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2278236'] for port, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:07.209487 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-86f6442c-a19f-42dd-a0e7-d7c92b82a633 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:10:07.314035 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event updated for resources: {'Port': ['ID=0ebe55f9-2a7a-4aba-87b1-b26f07199cdc,revision_number=2']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:10:07.318107 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:10:07.327069 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-507660'] for floatingip, after_update {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:07.491414 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Floating IP d1411a6c-d95a-405a-8684-ee9b6ad9855d associated. External IP: 172.27.1.99, port: 53f9e6bd-0116-4e65-95a5-b197d29610cf. May 31 12:10:07.933532 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:10:08.111727 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:10:08.121230 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:10:08.131372 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:10:08.131988 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:09:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:10:08.184764 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:08.246108 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-7a9fe360-9270-426e-95b2-533dbbe5c12b tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 2.6263092 May 31 12:10:08.246854 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:10:08.249199 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961" -d '{"events": [{"name": "network-changed", "server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:10:08.291986 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:10:08 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-b5dc159f-e3f1-41ed-b598-7012955ea2bc x-openstack-request-id: req-b5dc159f-e3f1-41ed-b598-7012955ea2bc {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:10:08.292309 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:10:08.292593 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-b5dc159f-e3f1-41ed-b598-7012955ea2bc {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:10:08.293018 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'status': 'completed', 'code': 200} May 31 12:10:08.357664 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:10:08.387345 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-4ea395ea-3588-4c5b-bc34-efa60c1bd982 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:08.388898 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-4ea395ea-3588-4c5b-bc34-efa60c1bd982 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 1098 time: 0.0740843 May 31 12:10:08.520616 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-bd775321-3d3b-4478-9070-7999f26f6a50 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:08.522105 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-bd775321-3d3b-4478-9070-7999f26f6a50 service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.1273005 May 31 12:10:08.625270 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-2612ff10-40db-4a3d-891f-c4b4dc385e96 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=53f9e6bd-0116-4e65-95a5-b197d29610cf HTTP/1.1" status: 200 len: 998 time: 0.0962713 May 31 12:10:08.686780 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-9d865618-b3d7-4726-bd91-19c214f7e556 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:08.688197 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-9d865618-b3d7-4726-bd91-19c214f7e556 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0573292 May 31 12:10:08.708080 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:10Z', 'revision_number': 1, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}, {'id': 'd1411a6c-d95a-405a-8684-ee9b6ad9855d', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.99', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'fixed_ip_address': '172.20.0.8', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:f2:ac:66', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d094ad5-4278-4dce-ac78-413a587effbc', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:10:06Z', 'updated_at': '2021-05-31T12:10:06Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:10:08.711903 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:10:08.720724 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:10:08.721327 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:09:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:10:08.768182 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-3a6ec6f6-0f0f-47df-b9b0-20abd7b688b6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:08.769234 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-3a6ec6f6-0f0f-47df-b9b0-20abd7b688b6 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0749748 May 31 12:10:08.901093 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-51bda0fb-f375-441c-9686-ec35009f8c4e service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1252310 May 31 12:10:08.903837 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:10:09.033297 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-c0c65ea6-4d69-4d34-94c5-4e70ddc31961 req-1f7f399a-25b7-46f8-a722-fc9d4ac0857f service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1254890 May 31 12:10:09.251259 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:10Z', 'revision_number': 1, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}, {'id': 'd1411a6c-d95a-405a-8684-ee9b6ad9855d', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.99', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'fixed_ip_address': '172.20.0.8', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:f2:ac:66', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d094ad5-4278-4dce-ac78-413a587effbc', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:10:06Z', 'updated_at': '2021-05-31T12:10:06Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:10:09.255219 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d21720f4-cb00-49d3-bf10-1d4c9fef5fe7 None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:10Z', 'revision_number': 1, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}, {'id': 'd1411a6c-d95a-405a-8684-ee9b6ad9855d', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.99', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'fixed_ip_address': '172.20.0.8', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:f2:ac:66', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d094ad5-4278-4dce-ac78-413a587effbc', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:10:06Z', 'updated_at': '2021-05-31T12:10:06Z', 'revision_number': 0, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:10:09.427629 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c84d9505-30dc-43e9-a6e6-78252e0c5369 None None] New status for floating IP d1411a6c-d95a-405a-8684-ee9b6ad9855d: ACTIVE {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:10:09.446727 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:20.892583 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dc7df4ec-0ab4-4ba6-9fff-4e4d0e64a7f1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:23.331867 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:23.411673 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-588240ec-ca3a-43f0-b0a7-39988521e27e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:24.640720 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:26.131776 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:39.433750 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:41.456105 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:10:48.983684 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3166ddac-d07c-417b-b4fb-0633220de15a req-a455e579-111b-4ff7-8a68-b2c09c7f518a service neutron] 10.222.0.47 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0794506 May 31 12:10:49.123852 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3166ddac-d07c-417b-b4fb-0633220de15a req-2ec1d54b-3a5c-4d56-904e-c4edbf0969c5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:49.125288 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3166ddac-d07c-417b-b4fb-0633220de15a req-2ec1d54b-3a5c-4d56-904e-c4edbf0969c5 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1098 time: 0.0737948 May 31 12:10:49.268464 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3166ddac-d07c-417b-b4fb-0633220de15a req-630e89d4-50b7-43fc-9193-f54fe9ea1651 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:49.270469 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3166ddac-d07c-417b-b4fb-0633220de15a req-630e89d4-50b7-43fc-9193-f54fe9ea1651 service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.1260335 May 31 12:10:49.466586 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3166ddac-d07c-417b-b4fb-0633220de15a req-35c33014-e661-4d46-9e8a-3911feb6a102 service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43 HTTP/1.1" status: 200 len: 1000 time: 0.1901248 May 31 12:10:49.532285 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3166ddac-d07c-417b-b4fb-0633220de15a req-bf3183b5-f705-43c2-b5e2-08bb71ff8b2a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:49.532285 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3166ddac-d07c-417b-b4fb-0633220de15a req-bf3183b5-f705-43c2-b5e2-08bb71ff8b2a service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0576265 May 31 12:10:49.604607 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3166ddac-d07c-417b-b4fb-0633220de15a req-7de71151-bc9d-4f94-b8a2-70b4a369095a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:10:49.606104 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3166ddac-d07c-417b-b4fb-0633220de15a req-7de71151-bc9d-4f94-b8a2-70b4a369095a service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0700989 May 31 12:10:49.736769 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3166ddac-d07c-417b-b4fb-0633220de15a req-93fdaccc-2bd1-4720-8ba3-36fee3ac9688 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1245861 May 31 12:10:49.868066 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-3166ddac-d07c-417b-b4fb-0633220de15a req-b4937b17-4ef1-4138-add6-86a17d87e1b5 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1254778 May 31 12:10:50.907607 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-9067dc82-dac1-4c30-822b-0b04dac1e7b9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:53.331162 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:53.427138 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f52023f8-d904-43fb-988b-ca2f062eb092 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:54.638818 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:10:56.164200 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:09.432395 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:18.454206 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:11:20.923877 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-8876bf42-7e03-4c0c-b568-4fb852e3bfd4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:23.318455 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:23.437682 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-cc08375f-a93c-4a33-9047-7d1a7b955766 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:24.672242 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:26.169147 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:39.483120 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:47.227087 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40954) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:11:47.487821 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Deleting port 0ebe55f9-2a7a-4aba-87b1-b26f07199cdc {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:11:47.488063 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:47.610863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:47.611734 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:47.736108 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Calling delete_port for 0ebe55f9-2a7a-4aba-87b1-b26f07199cdc owned by network:floatingip {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:11:47.934432 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:47.938472 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-fb25f541-4483-4df1-a101-04fb4ba40c78 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:11:47.984441 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event deleted for resources: {'Port': ['ID=0ebe55f9-2a7a-4aba-87b1-b26f07199cdc,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:11:48.022666 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.023039 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.023403 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.023768 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.024114 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.024515 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.024867 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.025220 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.025585 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.026003 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.026483 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.026848 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99040) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:11:48.035800 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:11:48.114037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:48.114426 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Floating IP d1411a6c-d95a-405a-8684-ee9b6ad9855d disassociated (deleted). External IP: 172.27.1.99, port: 53f9e6bd-0116-4e65-95a5-b197d29610cf. May 31 12:11:48.587346 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:11:48.788847 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:11:48.797962 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:11:48.807556 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:11:48.808267 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:11:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:11:48.877008 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:48.937686 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "DELETE /v2.0/floatingips/d1411a6c-d95a-405a-8684-ee9b6ad9855d HTTP/1.1" status: 204 len: 168 time: 1.7094328 May 31 12:11:48.938349 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:11:48.940652 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-514a7ed1-72e9-421f-a757-b6c9746d0bcf" -d '{"events": [{"name": "network-changed", "server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:11:48.989115 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:11:49.034696 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:11:48 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-58fa5408-8542-475c-8308-e1461cb8e4f8 x-openstack-request-id: req-58fa5408-8542-475c-8308-e1461cb8e4f8 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:11:49.035037 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:11:49.035330 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-58fa5408-8542-475c-8308-e1461cb8e4f8 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:11:49.035776 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'status': 'completed', 'code': 200} May 31 12:11:49.137463 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-278964ef-f57f-4300-b27d-cc14c0e222a1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:49.138949 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-278964ef-f57f-4300-b27d-cc14c0e222a1 service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 1098 time: 0.0804765 May 31 12:11:49.274594 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-24119fc7-1e3a-495f-b17d-59cc2ded5345 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:49.276039 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-24119fc7-1e3a-495f-b17d-59cc2ded5345 service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.1307464 May 31 12:11:49.347851 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-ea843288-49ac-42ec-93cd-1a35ab8644cb service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=53f9e6bd-0116-4e65-95a5-b197d29610cf HTTP/1.1" status: 200 len: 217 time: 0.0652974 May 31 12:11:49.356498 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:10Z', 'revision_number': 1, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:11:49.360461 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:11:49.369303 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:11:49.370000 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:11:23Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:11:49.418027 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-7aea63b5-d4f6-4fd6-a917-28935db01d07 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:49.419376 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-7aea63b5-d4f6-4fd6-a917-28935db01d07 service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0642231 May 31 12:11:49.491332 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9e0a93f-5ac5-48be-a3c1-29616f78d990 req-e69ff092-dd4c-45f9-b7d6-f37e519580a4 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:49.492900 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a9e0a93f-5ac5-48be-a3c1-29616f78d990 req-e69ff092-dd4c-45f9-b7d6-f37e519580a4 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 945 time: 0.0817919 May 31 12:11:49.506717 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-44b439ba-ab7b-4c67-904d-a5835fcf907f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:49.508313 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-44b439ba-ab7b-4c67-904d-a5835fcf907f service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0812235 May 31 12:11:49.550058 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-a9e0a93f-5ac5-48be-a3c1-29616f78d990 req-57bbcaca-0dc9-4bc0-9f33-dc724e6fa885 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0506680 May 31 12:11:49.553830 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:11:49.657203 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-c2b110bf-a57c-4dc5-8d60-dc8105b1bd64 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1396372 May 31 12:11:49.788904 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-514a7ed1-72e9-421f-a757-b6c9746d0bcf req-7da7f656-721e-4d2f-b1bc-39d70654af35 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1248918 May 31 12:11:49.898778 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:10Z', 'revision_number': 1, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:11:49.902627 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e9fcf08a-f6b9-4eb9-9f62-7c740cc46a99 None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_floatingips': [{'id': '0cea0cc6-fc31-47d9-a884-be7ea8ab0e58', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'floating_ip_address': '172.27.1.78', 'floating_network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'router_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'port_id': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'fixed_ip_address': '172.20.0.3', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'mac_address': 'fa:16:3e:82:1d:4b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7f1c6589-297d-44de-9f53-77c11b989767', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-31T12:08:07Z', 'updated_at': '2021-05-31T12:08:10Z', 'revision_number': 1, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:11:50.104599 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-06a30519-a20a-4cd2-9f52-4683f62d140d None None] New status for floating IP d1411a6c-d95a-405a-8684-ee9b6ad9855d: DOWN {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:11:50.733360 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56ce40ca-b255-4f85-bede-3f794eeea1cd req-466cf5c8-2460-47bb-8196-00e4f19e3f1f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:50.734065 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-56ce40ca-b255-4f85-bede-3f794eeea1cd req-466cf5c8-2460-47bb-8196-00e4f19e3f1f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:11:50.802255 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-56ce40ca-b255-4f85-bede-3f794eeea1cd req-466cf5c8-2460-47bb-8196-00e4f19e3f1f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:11:50.803466 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-56ce40ca-b255-4f85-bede-3f794eeea1cd req-466cf5c8-2460-47bb-8196-00e4f19e3f1f tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 945 time: 0.1521356 May 31 12:11:50.856090 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-56ce40ca-b255-4f85-bede-3f794eeea1cd req-6d7c0731-d825-4325-b85e-1d774a84ace0 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0469286 May 31 12:11:50.916779 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-b8c642ad-4823-48fc-a79e-254728a8be34 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:51.166875 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-fb126771-804d-425c-bdb5-59aee22888ff tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:51.168303 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-fb126771-804d-425c-bdb5-59aee22888ff tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 945 time: 0.0835128 May 31 12:11:51.303843 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Deleting port 53f9e6bd-0116-4e65-95a5-b197d29610cf {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:11:51.304174 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:51.527213 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] For port 53f9e6bd-0116-4e65-95a5-b197d29610cf, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=53f9e6bd-0116-4e65-95a5-b197d29610cf,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:11:51.527567 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:51.652316 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Calling delete_port for 53f9e6bd-0116-4e65-95a5-b197d29610cf owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:11:51.820084 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:51.825482 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-023696e9-f757-4fb7-817f-f90a919681d0 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:11:51.873545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event deleted for resources: {'Port': ['ID=53f9e6bd-0116-4e65-95a5-b197d29610cf,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:11:51.910405 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:11:51.954054 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] No DVR routers for this DVR port 53f9e6bd-0116-4e65-95a5-b197d29610cf on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:11:51.956737 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:51.959540 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'name': 'network-vif-deleted', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:11:51.961816 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-872e3996-822e-4777-a2e8-0f7c3787fd54" -d '{"events": [{"server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "name": "network-vif-deleted", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:11:52.010640 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:11:51 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-9f6ffb90-ca8b-42b8-ba4c-1accd52b52f6 x-openstack-request-id: req-9f6ffb90-ca8b-42b8-ba4c-1accd52b52f6 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:11:52.010970 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "0d094ad5-4278-4dce-ac78-413a587effbc", "name": "network-vif-deleted", "tag": "53f9e6bd-0116-4e65-95a5-b197d29610cf", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:11:52.011262 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-9f6ffb90-ca8b-42b8-ba4c-1accd52b52f6 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:11:52.011699 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0d094ad5-4278-4dce-ac78-413a587effbc', 'name': 'network-vif-deleted', 'tag': '53f9e6bd-0116-4e65-95a5-b197d29610cf', 'status': 'completed', 'code': 200} May 31 12:11:52.037201 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-0a0a95e2-605a-41a0-a60b-009791481ee9 req-259f34c2-b97b-4a7b-89e3-0816dc1602b8 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 210 time: 0.0440772 May 31 12:11:52.077658 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-f71e1cc5-54df-45d7-a25a-b9ebc16b8cac req-7acc17cd-fb3a-473d-9f9d-4e1171f63424 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "DELETE /v2.0/ports/53f9e6bd-0116-4e65-95a5-b197d29610cf HTTP/1.1" status: 204 len: 173 time: 0.9023292 May 31 12:11:53.203366 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-db1b08f6-ec74-4214-985c-69ffe3a8f374 req-dcf5e965-0837-4089-bd25-f97e23c966e1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:11:53.259668 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-db1b08f6-ec74-4214-985c-69ffe3a8f374 req-dcf5e965-0837-4089-bd25-f97e23c966e1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:11:53.260925 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-db1b08f6-ec74-4214-985c-69ffe3a8f374 req-dcf5e965-0837-4089-bd25-f97e23c966e1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=0d094ad5-4278-4dce-ac78-413a587effbc HTTP/1.1" status: 200 len: 210 time: 0.0994880 May 31 12:11:53.339815 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:53.454050 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6ba0972c-838a-4c00-98fd-0381d9044a41 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:54.677073 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:55.452389 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:11:55.618815 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 53f9e6bd-0116-4e65-95a5-b197d29610cf no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:11:55.656566 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: 53f9e6bd-0116-4e65-95a5-b197d29610cf {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:11:55.656915 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device 53f9e6bd-0116-4e65-95a5-b197d29610cf not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:11:56.163148 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:57.015245 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 40990) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:11:57.277691 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Deleting port 1eb0c86e-5c25-436c-bf8e-b066b71f4d19 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:11:57.277932 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:57.399717 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:57.400036 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:57.524393 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Calling delete_port for 1eb0c86e-5c25-436c-bf8e-b066b71f4d19 owned by network:floatingip {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:11:57.705718 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:57.709909 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-b0794eea-f272-450e-acdf-dfda16e12b97 tempest-TestShelveInstance-62423705 tempest-TestShelveInstance-62423705-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:11:57.755917 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event deleted for resources: {'Port': ['ID=1eb0c86e-5c25-436c-bf8e-b066b71f4d19,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:11:57.758995 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:11:57.844268 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:57.844579 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.db.l3_db [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Floating IP 0cea0cc6-fc31-47d9-a884-be7ea8ab0e58 disassociated (deleted). External IP: 172.27.1.78, port: dc081ad4-14a2-4ac7-8d8e-3b836037dc43. May 31 12:11:58.282049 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:11:58.458875 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:11:58.469448 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:11:58.478861 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:11:58.479495 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:11:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:11:58.544142 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:11:58.599274 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-7402d452-4b25-49db-84c9-14735658ed10 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "DELETE /v2.0/floatingips/0cea0cc6-fc31-47d9-a884-be7ea8ab0e58 HTTP/1.1" status: 204 len: 168 time: 1.5829499 May 31 12:11:58.600703 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:11:58.603041 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-cc344193-51a5-4453-9826-b9bc3e1c7df7" -d '{"events": [{"name": "network-changed", "server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:11:58.648994 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 179 Content-Type: application/json Date: Mon, 31 May 2021 12:11:58 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-73d683ff-b12b-4b30-a26f-7a05bf2ca60e x-openstack-request-id: req-73d683ff-b12b-4b30-a26f-7a05bf2ca60e {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:11:58.649603 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"name": "network-changed", "server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:11:58.650100 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-73d683ff-b12b-4b30-a26f-7a05bf2ca60e {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:11:58.650819 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'status': 'completed', 'code': 200} May 31 12:11:58.711458 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:11:58.752056 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-b5c26fb7-f855-41af-8f0b-89b3d8bd6aff service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:58.753949 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-b5c26fb7-f855-41af-8f0b-89b3d8bd6aff service neutron] 10.222.0.47 "GET /v2.0/ports?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 1098 time: 0.0795333 May 31 12:11:58.887771 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-2c0923e3-de93-44ee-8b47-dff2c90408e1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:58.889197 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-2c0923e3-de93-44ee-8b47-dff2c90408e1 service neutron] 10.222.0.47 "GET /v2.0/networks?id=8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 200 len: 912 time: 0.1291103 May 31 12:11:58.963802 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-c85dad46-0c1e-41bd-a5e5-dba838ef043b service neutron] 10.222.0.47 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43 HTTP/1.1" status: 200 len: 217 time: 0.0676866 May 31 12:11:59.023923 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:11:59.027951 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:11:59.036628 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:11:59.038128 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:11:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:11:59.043917 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-9acf3162-a6fd-4eaa-a697-a68b090db9cc service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:59.045400 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-9acf3162-a6fd-4eaa-a697-a68b090db9cc service neutron] 10.222.0.47 "GET /v2.0/subnets?id=4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 200 len: 850 time: 0.0748651 May 31 12:11:59.123575 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-b4268027-6541-47ad-a236-f28cba5f74f4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:59.125045 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-b4268027-6541-47ad-a236-f28cba5f74f4 service neutron] 10.222.0.47 "GET /v2.0/ports?network_id=8a4efbb8-aa93-49be-a073-41ce66e25d2f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0723586 May 31 12:11:59.210030 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a562340-aeeb-48a4-8b52-6306f74029ec req-036bc644-7954-42e2-917b-4be0fb05e553 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99039) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:11:59.211533 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1a562340-aeeb-48a4-8b52-6306f74029ec req-036bc644-7954-42e2-917b-4be0fb05e553 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 945 time: 0.0897911 May 31 12:11:59.223390 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:11:59.262661 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [req-1a562340-aeeb-48a4-8b52-6306f74029ec req-e12baca1-7179-4bbe-b826-25945d124826 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0430822 May 31 12:11:59.265489 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-8444659e-88a9-496b-9e86-4f80f0a19b91 service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1320753 May 31 12:11:59.399069 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-cc344193-51a5-4453-9826-b9bc3e1c7df7 req-a7f95cd9-0896-445d-9cd7-2d0f077abe4e service neutron] 10.222.0.47 "GET /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1249990 May 31 12:11:59.535631 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:11:59.539605 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-93d3b497-32af-4c93-a5c6-fe369e40388c None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:52Z', 'revision_number': 4, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi', '_interfaces': [{'id': '077741d4-ebe8-4cb3-9d13-97cd5c09d04c', 'name': '', 'network_id': '8a4efbb8-aa93-49be-a073-41ce66e25d2f', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'mac_address': 'fa:16:3e:2c:99:83', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_interface', 'standard_attr_id': 652, 'fixed_ips': [{'subnet_id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:52Z', 'updated_at': '2021-05-31T12:04:59Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'subnets': [{'id': '4a96f417-9ce3-448a-9f7f-529c69a0ace7', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:11:59.741492 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-48693c28-2af1-4550-bc76-90d871756f6f None None] New status for floating IP 0cea0cc6-fc31-47d9-a884-be7ea8ab0e58: DOWN {{(pid=99042) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 31 12:12:00.457786 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc37fd03-0f94-4422-8e8b-5791a22c373d req-345cfd1d-d4c8-4179-853a-0c431acaefda tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:12:00.458466 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-bc37fd03-0f94-4422-8e8b-5791a22c373d req-345cfd1d-d4c8-4179-853a-0c431acaefda tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:12:00.528218 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-bc37fd03-0f94-4422-8e8b-5791a22c373d req-345cfd1d-d4c8-4179-853a-0c431acaefda tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:12:00.529375 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-bc37fd03-0f94-4422-8e8b-5791a22c373d req-345cfd1d-d4c8-4179-853a-0c431acaefda tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 945 time: 0.1533022 May 31 12:12:00.576276 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-bc37fd03-0f94-4422-8e8b-5791a22c373d req-ad2a0142-bb7f-47bc-8f0d-a4f0968d8a55 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/security-groups?id=bd77d026-8f31-4ec6-a2ff-f8e6a6857859&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0411670 May 31 12:12:00.658799 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-526c416e-4b16-4628-a781-2862ba3b7201 req-598eb0ab-5d29-4141-b139-b490512ff3c1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99040) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 31 12:12:00.660237 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-526c416e-4b16-4628-a781-2862ba3b7201 req-598eb0ab-5d29-4141-b139-b490512ff3c1 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 945 time: 0.0788493 May 31 12:12:00.794083 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Deleting port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:00.794418 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:01.016508 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] For port dc081ad4-14a2-4ac7-8d8e-3b836037dc43, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:01.016855 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:01.141016 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Calling delete_port for dc081ad4-14a2-4ac7-8d8e-3b836037dc43 owned by compute:nova {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:01.294099 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:01.301525 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-f7df8b50-e6cb-465d-8953-711e5d1fa7ed admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:01.345859 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event deleted for resources: {'Port': ['ID=dc081ad4-14a2-4ac7-8d8e-3b836037dc43,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:01.348806 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:01.421406 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] No DVR routers for this DVR port dc081ad4-14a2-4ac7-8d8e-3b836037dc43 on host hv-cinder-79123 {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:12:01.423824 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for port, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:01.426294 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'name': 'network-vif-deleted', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43'}] {{(pid=99040) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 31 12:12:01.428542 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST http://10.222.0.48/compute/v2.1/os-server-external-events -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8add5a6567bea01db760109e3dcb2172f4240899c7a56e5edba0ee3f761a6e05" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-OpenStack-Request-ID: req-89545e06-2aef-46dd-a564-7033835f6449" -d '{"events": [{"server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "name": "network-vif-deleted", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43"}]}' {{(pid=99040) _http_log_request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:519}} May 31 12:12:01.470570 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP: [200] Connection: close Content-Length: 183 Content-Type: application/json Date: Mon, 31 May 2021 12:12:01 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.41 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-f09086c0-8e5a-4862-a258-60c7c68e7a96 x-openstack-request-id: req-f09086c0-8e5a-4862-a258-60c7c68e7a96 {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:550}} May 31 12:12:01.471177 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] RESP BODY: {"events": [{"server_uuid": "7f1c6589-297d-44de-9f53-77c11b989767", "name": "network-vif-deleted", "tag": "dc081ad4-14a2-4ac7-8d8e-3b836037dc43", "status": "completed", "code": 200}]} {{(pid=99040) _http_log_response /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:582}} May 31 12:12:01.471177 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG novaclient.v2.client [-] POST call to compute for http://10.222.0.48/compute/v2.1/os-server-external-events used request id req-f09086c0-8e5a-4862-a258-60c7c68e7a96 {{(pid=99040) request /usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py:954}} May 31 12:12:01.471606 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '7f1c6589-297d-44de-9f53-77c11b989767', 'name': 'network-vif-deleted', 'tag': 'dc081ad4-14a2-4ac7-8d8e-3b836037dc43', 'status': 'completed', 'code': 200} May 31 12:12:01.504710 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-526c416e-4b16-4628-a781-2862ba3b7201 req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.47 "DELETE /v2.0/ports/dc081ad4-14a2-4ac7-8d8e-3b836037dc43 HTTP/1.1" status: 204 len: 173 time: 0.8389287 May 31 12:12:01.733422 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-47fa7224-eea2-4cdc-acd3-26624aa00887 req-2bd5b11b-07d9-403c-ad12-0605d1d27bfc tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Synchronizing usage tracker for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 31 12:12:01.835674 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.quota.resource [req-47fa7224-eea2-4cdc-acd3-26624aa00887 req-2bd5b11b-07d9-403c-ad12-0605d1d27bfc tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Unset dirty status for tenant:0e2c6e0e5c334c60a47f2dc2d0c79653 on resource:port {{(pid=99040) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 31 12:12:01.836832 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [req-47fa7224-eea2-4cdc-acd3-26624aa00887 req-2bd5b11b-07d9-403c-ad12-0605d1d27bfc tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "GET /v2.0/ports?device_id=7f1c6589-297d-44de-9f53-77c11b989767 HTTP/1.1" status: 200 len: 210 time: 0.1522267 May 31 12:12:02.931302 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 41008) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:03.145298 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Deleting port 6130415e-0d67-42e4-9395-9bb035dddeea {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:03.145668 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:03.263469 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:03.263791 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:03.386787 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Calling delete_port for 6130415e-0d67-42e4-9395-9bb035dddeea owned by network:floatingip {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:03.556976 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:03.560861 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-6cf20738-2224-45b7-b625-4067aa7334e0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:03.603912 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event deleted for resources: {'Port': ['ID=6130415e-0d67-42e4-9395-9bb035dddeea,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:03.634084 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device dc081ad4-14a2-4ac7-8d8e-3b836037dc43 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:12:03.642959 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.643329 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.643691 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.644042 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.644393 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.644746 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.645231 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.645444 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.645814 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.646722 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.647104 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.647447 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99039) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:03.655501 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:03.668941 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] No Port match for: dc081ad4-14a2-4ac7-8d8e-3b836037dc43 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:12:03.669290 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Device dc081ad4-14a2-4ac7-8d8e-3b836037dc43 not bound to the agent host hv-cinder-79123 {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:12:03.707832 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for floatingip, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:03.784603 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-123460'] for floatingip, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:03.785727 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c033f828-86cc-4f58-b6f7-5a1ed75d7a50 tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "DELETE /v2.0/floatingips/e64c4829-1a2c-4e4b-832d-03e4b32f81f4 HTTP/1.1" status: 204 len: 168 time: 0.8533978 May 31 12:12:05.279903 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41022) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:05.351136 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d66719c5-3b43-41a5-b73c-a5b5914f8a9e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:05.380082 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d66719c5-3b43-41a5-b73c-a5b5914f8a9e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:05.499415 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d66719c5-3b43-41a5-b73c-a5b5914f8a9e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:05.504340 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-66bb5085-861c-48d0-9219-93f856626fc2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:05.519163 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d66719c5-3b43-41a5-b73c-a5b5914f8a9e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=bd77d026-8f31-4ec6-a2ff-f8e6a6857859,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:05.522395 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d66719c5-3b43-41a5-b73c-a5b5914f8a9e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:05.578701 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d66719c5-3b43-41a5-b73c-a5b5914f8a9e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:05.579830 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-d66719c5-3b43-41a5-b73c-a5b5914f8a9e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] 10.222.0.48 "DELETE /v2.0/security-groups/bd77d026-8f31-4ec6-a2ff-f8e6a6857859 HTTP/1.1" status: 204 len: 168 time: 0.2989628 May 31 12:12:05.678114 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41028) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:06.352099 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:06.846039 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Deleting port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:06.846374 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:06.965716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] For port 077741d4-ebe8-4cb3-9d13-97cd5c09d04c, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:06.966053 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:07.086419 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Calling delete_port for 077741d4-ebe8-4cb3-9d13-97cd5c09d04c owned by network:router_interface {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:07.320968 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:07.324798 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-594a905c-9630-45e3-8ae9-599d624de4eb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:07.369264 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Pushing event deleted for resources: {'Port': ['ID=077741d4-ebe8-4cb3-9d13-97cd5c09d04c,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:07.371754 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:07.636216 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:07.985075 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Router 5b6f052e-0b0b-4603-9141-bb2a360accdf has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:12:08.262215 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:12:08.270678 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] Sync routers for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:12:08.274119 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-861486bf-6817-46ae-96a7-69585277f771 admin admin] 10.222.0.48 "PUT /v2.0/routers/5b6f052e-0b0b-4603-9141-bb2a360accdf/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.5949574 May 31 12:12:08.276653 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41034) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:08.281229 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:12:08.281867 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:11:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:08.452387 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks [] for router, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:08.459761 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: dv-cinder-791239-4-iscsi {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:12:08.627212 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:08.789065 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Deleting port 3410a2bc-9d4c-4de9-b287-f0317f808672 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:08.789744 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:08.797785 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66'}], 'enable_snat': True}, 'gw_port_id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:12:07Z', 'revision_number': 5, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port': {'id': '3410a2bc-9d4c-4de9-b287-f0317f808672', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:2c:0f:11', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 651, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.66', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:04:55Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:08.801584 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] Checking router: 5b6f052e-0b0b-4603-9141-bb2a360accdf for host: dv-cinder-791239-4-iscsi {{(pid=99041) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:12:08.810346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] Routers {'5b6f052e-0b0b-4603-9141-bb2a360accdf'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99041) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:12:08.810958 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:11:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['5b6f052e-0b0b-4603-9141-bb2a360accdf'], 'active': True} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:08.914252 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] For port 3410a2bc-9d4c-4de9-b287-f0317f808672, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=3410a2bc-9d4c-4de9-b287-f0317f808672,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:08.914590 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:08.960799 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvr_db [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] Set router 5b6f052e-0b0b-4603-9141-bb2a360accdf gateway port host: None {{(pid=99041) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:12:09.042343 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Calling delete_port for 3410a2bc-9d4c-4de9-b287-f0317f808672 owned by network:router_gateway {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:09.071215 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_hamode_db [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:12:08Z', 'revision_number': 6, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port_host': None}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99041) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:09.076836 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a5dad77f-dba1-41aa-b671-c02cb8a2e206 None None] The sync data for ids ['5b6f052e-0b0b-4603-9141-bb2a360accdf'] in dv-cinder-791239-4-iscsi is: [{'id': '5b6f052e-0b0b-4603-9141-bb2a360accdf', 'name': 'tempest-TestVolumeBootPattern-340341230-router', 'tenant_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:48Z', 'updated_at': '2021-05-31T12:12:08Z', 'revision_number': 6, 'project_id': '0e2c6e0e5c334c60a47f2dc2d0c79653', 'gw_port_host': None}] {{(pid=99041) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:12:09.276764 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.280545 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-2dc47ad6-7b9f-45c6-9861-0bfae76875f0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:09.324834 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Pushing event deleted for resources: {'Port': ['ID=3410a2bc-9d4c-4de9-b287-f0317f808672,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:09.327270 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:09.421484 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.452283 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.596331 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.598503 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.704770 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.705276 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 5b6f052e-0b0b-4603-9141-bb2a360accdf {{(pid=99040) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 12:12:09.718619 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] Notify callbacks [] for router, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.722248 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-c0355b59-993d-4efa-8f84-031ec5175d18 admin admin] 10.222.0.48 "DELETE /v2.0/routers/5b6f052e-0b0b-4603-9141-bb2a360accdf HTTP/1.1" status: 204 len: 168 time: 1.4421737 May 31 12:12:09.723308 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 41038) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:09.791732 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 3410a2bc-9d4c-4de9-b287-f0317f808672 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:12:09.792596 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.db_base_plugin_v2 [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] Deleting subnet 4a96f417-9ce3-448a-9f7f-529c69a0ace7 {{(pid=99039) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:12:09.795898 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.829305 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 3410a2bc-9d4c-4de9-b287-f0317f808672 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:12:09.829682 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 3410a2bc-9d4c-4de9-b287-f0317f808672 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:12:09.830103 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 077741d4-ebe8-4cb3-9d13-97cd5c09d04c no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:12:09.845438 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3122800'] for subnet, precommit_delete_associations {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:09.866462 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 077741d4-ebe8-4cb3-9d13-97cd5c09d04c {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:12:09.866819 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 077741d4-ebe8-4cb3-9d13-97cd5c09d04c not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:12:09.922451 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:10.334329 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2338065', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2330291'] for subnet, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:10.337945 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-55eb8cf6-b2c2-44fd-8f4e-a667c371724d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:10.372207 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=4a96f417-9ce3-448a-9f7f-529c69a0ace7,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:10.374779 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:10.377502 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:10.383007 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0abca86d-5ce8-4782-a7fc-01b2e491ef71 None None] Network 8a4efbb8-aa93-49be-a073-41ce66e25d2f requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:12:10.384336 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-abf425c3-6847-47e5-89b0-57508ecb39e9 admin admin] 10.222.0.48 "DELETE /v2.0/subnets/4a96f417-9ce3-448a-9f7f-529c69a0ace7 HTTP/1.1" status: 204 len: 168 time: 0.6592124 May 31 12:12:10.386914 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41042) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:10.596931 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:10.640296 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] DHCP port deletion for 8a4efbb8-aa93-49be-a073-41ce66e25d2f request from dv-cinder-791239-4-iscsi {{(pid=99041) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 31 12:12:10.669774 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Deleting port b19fb3d5-4dc8-4350-871f-7964e82205d8 {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:10.670109 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:10.706923 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Deleting port b19fb3d5-4dc8-4350-871f-7964e82205d8 {{(pid=99041) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:10.707309 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:10.770214 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] For port b19fb3d5-4dc8-4350-871f-7964e82205d8, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b19fb3d5-4dc8-4350-871f-7964e82205d8,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:10.770560 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:10.809476 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] For port b19fb3d5-4dc8-4350-871f-7964e82205d8, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=b19fb3d5-4dc8-4350-871f-7964e82205d8,segment=NetworkSegment(a29d566a-c32d-4472-972d-990571fdae77),segment_id=a29d566a-c32d-4472-972d-990571fdae77)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:10.809969 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:10.890892 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Calling delete_port for b19fb3d5-4dc8-4350-871f-7964e82205d8 owned by network:dhcp {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:10.932550 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Calling delete_port for b19fb3d5-4dc8-4350-871f-7964e82205d8 owned by network:dhcp {{(pid=99041) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:11.013862 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:11.018716 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-cf5d941a-a1f1-41f4-bdfe-f8c95d9a6c88 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:11.020061 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:11.023833 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-d766921c-4c58-4c0a-a8e3-265a90dd1995 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:11.065592 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Pushing event deleted for resources: {'Port': ['ID=b19fb3d5-4dc8-4350-871f-7964e82205d8,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:11.065855 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Pushing event deleted for resources: {'Port': ['ID=b19fb3d5-4dc8-4350-871f-7964e82205d8,revision_number=None']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:11.067399 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] No DVR routers for this DVR port b19fb3d5-4dc8-4350-871f-7964e82205d8 on host dv-cinder-791239-4-iscsi {{(pid=99040) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:12:11.069961 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:11.071243 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] No DVR routers for this DVR port b19fb3d5-4dc8-4350-871f-7964e82205d8 on host dv-cinder-791239-4-iscsi {{(pid=99041) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:12:11.077155 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.077499 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.077858 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.078187 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.078522 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.078838 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.079173 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.079478 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.079794 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.080363 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.080482 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.080699 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-4-iscsi {{(pid=99041) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 31 12:12:11.085420 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-ee5efa54-a0cf-4bd0-9502-392ea253a5d4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:11.183992 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:11.197304 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['8a4efbb8-aa93-49be-a073-41ce66e25d2f']}} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:11.209197 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'a29d566a-c32d-4472-972d-990571fdae77') {'for_net_delete': True} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:11.209491 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.services.segments.db [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'a29d566a-c32d-4472-972d-990571fdae77') {} {{(pid=99040) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:11.220853 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3096822'] for segment, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:11.236799 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:11.243766 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Releasing vlan 362 on physical network physnet1 to pool {{(pid=99040) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:12:11.244149 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2278144'] for segment, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: Traceback (most recent call last): May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: timer() May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: cb(*args, **kw) May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: return func(*args, **kwargs) May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: self._notify() May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: self.callback(batched_events) May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: event.method(event) May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: return f(self, *a, **k) May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._get(url).json() May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: return self.request(url, 'GET', **kwargs) May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:12:11.257947 dv-cinder-791239-4-iscsi neutron-server[99040]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:12:11.384791 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:11.389325 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:11.391761 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:11.414003 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Pushing event deleted for resources: {'Network': ['ID=8a4efbb8-aa93-49be-a073-41ce66e25d2f,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:11.416260 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:11.476801 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-7fa8a8ad-21d1-4213-b14b-849af9bb20b7 admin admin] 10.222.0.48 "DELETE /v2.0/networks/8a4efbb8-aa93-49be-a073-41ce66e25d2f HTTP/1.1" status: 204 len: 168 time: 1.0887911 May 31 12:12:11.479518 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41046) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:11.777292 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b19fb3d5-4dc8-4350-871f-7964e82205d8 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:12:11.811287 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: b19fb3d5-4dc8-4350-871f-7964e82205d8 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:12:11.811638 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device b19fb3d5-4dc8-4350-871f-7964e82205d8 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:12:12.006800 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-501566'] for router_interface, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:12.507520 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Deleting port 4a85e41c-d278-4239-b4f9-257929f10c1a {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:12.507856 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:12.627641 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] For port 4a85e41c-d278-4239-b4f9-257929f10c1a, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=4a85e41c-d278-4239-b4f9-257929f10c1a,segment=NetworkSegment(9bd3a04f-0739-4430-9c94-924934a7ab52),segment_id=9bd3a04f-0739-4430-9c94-924934a7ab52)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:12.627993 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:12.755789 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Calling delete_port for 4a85e41c-d278-4239-b4f9-257929f10c1a owned by network:router_interface {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:12.967300 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:12.970863 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-eef34503-139b-4b17-9ea1-b5657f0faa9c tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:13.016527 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Pushing event deleted for resources: {'Port': ['ID=4a85e41c-d278-4239-b4f9-257929f10c1a,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:13.048007 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:13.290685 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-556030', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-505262'] for router_interface, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:13.644093 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Router 0e144741-d108-4809-906b-0ffd45f625d3 has already been hosted by L3 agent 66e8cc85-710b-42e5-898a-e2ed7f990cc6 {{(pid=99040) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 31 12:12:13.918737 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] Notify agent at l3_agent.dv-cinder-791239-4-iscsi the message routers_updated {{(pid=99040) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 31 12:12:13.927460 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] Sync routers for ids ['0e144741-d108-4809-906b-0ffd45f625d3'] in dv-cinder-791239-4-iscsi {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 31 12:12:13.928077 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-d0adad20-66c7-4344-951b-2597a3b950bd admin admin] 10.222.0.48 "PUT /v2.0/routers/0e144741-d108-4809-906b-0ffd45f625d3/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.4475543 May 31 12:12:13.930564 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41050) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:13.936785 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] Routers {'0e144741-d108-4809-906b-0ffd45f625d3'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:12:13.937384 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:11:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0e144741-d108-4809-906b-0ffd45f625d3'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:14.102926 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks [] for router, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:14.109702 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] Set router 0e144741-d108-4809-906b-0ffd45f625d3 gateway port host: dv-cinder-791239-4-iscsi {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:12:14.282943 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:14.421319 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'external_fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112'}], 'enable_snat': True}, 'gw_port_id': '2cf42247-1a41-4760-8c77-b8388402439a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:12:12Z', 'revision_number': 5, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port': {'id': '2cf42247-1a41-4760-8c77-b8388402439a', 'name': '', 'network_id': '8a73d753-6759-4d77-a99f-016e97b0a680', 'tenant_id': '', 'mac_address': 'fa:16:3e:b7:9c:01', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0e144741-d108-4809-906b-0ffd45f625d3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'ip_address': '172.27.1.112', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-4-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-31T12:04:58Z', 'updated_at': '2021-05-31T12:05:05Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '4b97143c-47ad-4e8e-b97f-9970a81c7a23', 'cidr': '172.27.1.0/24', 'gateway_ip': '172.27.1.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-4-iscsi'}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:14.423295 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Deleting port 2cf42247-1a41-4760-8c77-b8388402439a {{(pid=99040) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:14.423629 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:14.425096 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] Checking router: 0e144741-d108-4809-906b-0ffd45f625d3 for host: dv-cinder-791239-4-iscsi {{(pid=99042) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 31 12:12:14.433530 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] Routers {'0e144741-d108-4809-906b-0ffd45f625d3'} bound to L3 agent in host dv-cinder-791239-4-iscsi {{(pid=99042) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 31 12:12:14.434148 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-4-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-31T11:46:23Z,description=,heartbeat_timestamp=2021-05-31T12:11:53Z,host='dv-cinder-791239-4-iscsi',id=66e8cc85-710b-42e5-898a-e2ed7f990cc6,load=0,resource_versions=,resources_synced=,started_at=2021-05-31T11:46:24Z,topic='l3_agent')) {'router_ids': ['0e144741-d108-4809-906b-0ffd45f625d3'], 'active': True} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:14.543181 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.db [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] For port 2cf42247-1a41-4760-8c77-b8388402439a, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=2cf42247-1a41-4760-8c77-b8388402439a,segment=NetworkSegment(ec17477a-1229-4e63-8304-a2f0dbd731dd),segment_id=ec17477a-1229-4e63-8304-a2f0dbd731dd)] {{(pid=99040) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:14.543433 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:14.585477 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_dvr_db [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] Set router 0e144741-d108-4809-906b-0ffd45f625d3 gateway port host: None {{(pid=99042) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 31 12:12:14.662774 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.plugins.ml2.plugin [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Calling delete_port for 2cf42247-1a41-4760-8c77-b8388402439a owned by network:router_gateway {{(pid=99040) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:14.688161 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.l3_hamode_db [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:12:14Z', 'revision_number': 6, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port_host': None}], 'dv-cinder-791239-4-iscsi', False) {} {{(pid=99042) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:14.691966 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fa723534-2739-45aa-9355-df87927c1c94 None None] The sync data for ids ['0e144741-d108-4809-906b-0ffd45f625d3'] in dv-cinder-791239-4-iscsi is: [{'id': '0e144741-d108-4809-906b-0ffd45f625d3', 'name': 'tempest-TestVolumeBootPattern-750418318-router', 'tenant_id': '34d7d5411875496f938fb5fb58f45223', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-31T12:04:57Z', 'updated_at': '2021-05-31T12:12:14Z', 'revision_number': 6, 'project_id': '34d7d5411875496f938fb5fb58f45223', 'gw_port_host': None}] {{(pid=99042) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 31 12:12:14.865265 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2278229'] for port, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:14.868801 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-0d3a9044-e4fa-45b3-8dea-15ce9650618e tempest-TestVolumeBootPattern-1024277422 tempest-TestVolumeBootPattern-1024277422-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:14.913009 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Pushing event deleted for resources: {'Port': ['ID=2cf42247-1a41-4760-8c77-b8388402439a,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:14.915413 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:15.007238 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-507683'] for router_gateway, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.282278 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-505774', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2155'] for router, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.284446 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.384726 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-504978'] for router, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.385177 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 0e144741-d108-4809-906b-0ffd45f625d3 {{(pid=99040) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 31 12:12:15.398708 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] Notify callbacks [] for router, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.399834 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-d12e5b59-9ea7-4309-a302-16c4455d5092 admin admin] 10.222.0.48 "DELETE /v2.0/routers/0e144741-d108-4809-906b-0ffd45f625d3 HTTP/1.1" status: 204 len: 168 time: 1.4683816 May 31 12:12:15.402391 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41054) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:15.460719 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.db.db_base_plugin_v2 [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] Deleting subnet 03af805c-fc50-465f-bbbf-187db4b13f93 {{(pid=99040) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 31 12:12:15.463616 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.515593 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3096829'] for subnet, precommit_delete_associations {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.596119 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1565137'] for subnet, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.790918 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 2cf42247-1a41-4760-8c77-b8388402439a no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:12:15.827932 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 2cf42247-1a41-4760-8c77-b8388402439a {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:12:15.828203 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 2cf42247-1a41-4760-8c77-b8388402439a not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:12:15.828632 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 4a85e41c-d278-4239-b4f9-257929f10c1a no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:12:15.861882 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 4a85e41c-d278-4239-b4f9-257929f10c1a {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:12:15.862242 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 4a85e41c-d278-4239-b4f9-257929f10c1a not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:12:15.920637 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1566820', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112787', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2111654', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2831123'] for subnet, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.925141 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:15.957539 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] Pushing event deleted for resources: {'Subnet': ['ID=03af805c-fc50-465f-bbbf-187db4b13f93,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:15.961739 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:15.963946 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for subnet, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:15.969141 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e24a4bbe-d718-466c-8aa3-dc8f0ea2482e None None] Network 219aba32-474c-4847-b9d2-5a8ddc5c5369 requested from dv-cinder-791239-4-iscsi {{(pid=99041) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 31 12:12:15.972238 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 41058) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:15.972902 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-ef909a7a-72f1-4303-b6a1-119cb7cb9d3e admin admin] 10.222.0.48 "DELETE /v2.0/subnets/03af805c-fc50-465f-bbbf-187db4b13f93 HTTP/1.1" status: 204 len: 168 time: 0.5688658 May 31 12:12:16.081534 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1574309'] for network, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.156150 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Deleting port 68599998-f433-42bf-80ae-c64a075f3712 {{(pid=99039) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:16.156494 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.208865 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] DHCP port deletion for 219aba32-474c-4847-b9d2-5a8ddc5c5369 request from dv-cinder-791239-4-iscsi {{(pid=99041) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 31 12:12:16.260723 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.db [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] For port 68599998-f433-42bf-80ae-c64a075f3712, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=68599998-f433-42bf-80ae-c64a075f3712,segment=NetworkSegment(9bd3a04f-0739-4430-9c94-924934a7ab52),segment_id=9bd3a04f-0739-4430-9c94-924934a7ab52)] {{(pid=99039) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:16.261070 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.306284 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] Deleting port 68599998-f433-42bf-80ae-c64a075f3712 {{(pid=99041) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 31 12:12:16.306662 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8759697375299'] for port, before_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.382809 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.plugin [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Calling delete_port for 68599998-f433-42bf-80ae-c64a075f3712 owned by network:dhcp {{(pid=99039) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:16.403583 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.db [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] For port 68599998-f433-42bf-80ae-c64a075f3712, host dv-cinder-791239-4-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-4-iscsi',level=0,port_id=68599998-f433-42bf-80ae-c64a075f3712,segment=NetworkSegment(9bd3a04f-0739-4430-9c94-924934a7ab52),segment_id=9bd3a04f-0739-4430-9c94-924934a7ab52)] {{(pid=99041) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 31 12:12:16.403972 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-506607'] for port, precommit_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.526293 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] Calling delete_port for 68599998-f433-42bf-80ae-c64a075f3712 owned by network:dhcp {{(pid=99041) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 31 12:12:16.527448 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2184930'] for port, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.532185 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1f2ebf56-b13a-4dd6-a985-4f6b708db278 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:16.570663 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1458890', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-112712', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-556012', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8759697376422', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8759696991708'] for port, after_delete {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.575111 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:16.577557 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Pushing event deleted for resources: {'Port': ['ID=68599998-f433-42bf-80ae-c64a075f3712,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:16.580685 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:16.583906 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] No DVR routers for this DVR port 68599998-f433-42bf-80ae-c64a075f3712 on host dv-cinder-791239-4-iscsi {{(pid=99039) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:12:16.622227 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] Pushing event deleted for resources: {'Port': ['ID=68599998-f433-42bf-80ae-c64a075f3712,revision_number=None']} {{(pid=99041) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:16.624024 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] No DVR routers for this DVR port 68599998-f433-42bf-80ae-c64a075f3712 on host dv-cinder-791239-4-iscsi {{(pid=99041) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 31 12:12:16.625391 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG oslo_concurrency.lockutils [None req-3f446088-702c-4daa-9bab-acf8cd23b8d8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99041) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:16.709559 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1567582', 'neutron.services.segments.db._delete_segments_for_network-8759697518300'] for network, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.722234 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['219aba32-474c-4847-b9d2-5a8ddc5c5369']}} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:16.734220 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '9bd3a04f-0739-4430-9c94-924934a7ab52') {'for_net_delete': True} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:16.734509 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.services.segments.db [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '9bd3a04f-0739-4430-9c94-924934a7ab52') {} {{(pid=99039) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 31 12:12:16.746912 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8759697527453', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3122811'] for segment, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.763383 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046'] for segment, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.769110 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Releasing vlan 370 on physical network physnet1 to pool {{(pid=99039) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 31 12:12:16.769479 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1571046', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2184887'] for segment, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: Traceback (most recent call last): May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: timer() May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: cb(*args, **kw) May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: return func(*args, **kwargs) May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: self._notify() May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: self.callback(batched_events) May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: event.method(event) May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_id = self._get_aggregate_id(event.segment_id) May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: aggregate_uuid = self.p_client.list_aggregates( May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: return f(self, *a, **k) May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._get(url).json() May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: return self._client.get(url, endpoint_filter=self._ks_filter, May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: return self.request(url, 'GET', **kwargs) May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: base_url = self.get_endpoint(auth, allow=allow, May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: auth = self._auth_required(auth, 'determine endpoint URL') May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 31 12:12:16.783234 dv-cinder-791239-4-iscsi neutron-server[99039]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 31 12:12:16.916062 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1567575', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-412494', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-556023', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-507710'] for network, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.919892 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-25c6a3ad-3f6f-4746-b531-e585c7d52ea9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:16.932834 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-555986'] for network, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:16.944258 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Pushing event deleted for resources: {'Network': ['ID=219aba32-474c-4847-b9d2-5a8ddc5c5369,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:16.947009 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] 10.222.0.48 "DELETE /v2.0/networks/219aba32-474c-4847-b9d2-5a8ddc5c5369 HTTP/1.1" status: 204 len: 168 time: 0.9736798 May 31 12:12:16.947838 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-c04c12ce-5066-4a89-89a0-c9a6194b4de5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:17.252131 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 41064) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:17.302574 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-00726a6f-e99d-4e65-9f47-6ecd6b94855d admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=0e2c6e0e5c334c60a47f2dc2d0c79653&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0486665 May 31 12:12:17.305207 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [-] (99039) accepted ('10.222.0.48', 41068) {{(pid=99039) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:17.362466 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1ca39e16-afd8-4fc3-b07f-307c127f7e85 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:17.387907 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1ca39e16-afd8-4fc3-b07f-307c127f7e85 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:17.571881 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1ca39e16-afd8-4fc3-b07f-307c127f7e85 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:17.575674 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-fd2589dd-b6db-41c4-b87c-6919ebdc9dc2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:17.591241 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1ca39e16-afd8-4fc3-b07f-307c127f7e85 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7174e6c9-cf73-4a54-b506-efa9bb4198e0,revision_number=None']} {{(pid=99039) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:17.598039 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG oslo_concurrency.lockutils [None req-1ca39e16-afd8-4fc3-b07f-307c127f7e85 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99039) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:17.600336 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron_lib.callbacks.manager [None req-1ca39e16-afd8-4fc3-b07f-307c127f7e85 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99039) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:17.601437 dv-cinder-791239-4-iscsi neutron-server[99039]: INFO neutron.wsgi [None req-1ca39e16-afd8-4fc3-b07f-307c127f7e85 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/7174e6c9-cf73-4a54-b506-efa9bb4198e0 HTTP/1.1" status: 204 len: 168 time: 0.2952914 May 31 12:12:17.778352 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 68599998-f433-42bf-80ae-c64a075f3712 no longer exists at agent ovs-agent-dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 31 12:12:17.811994 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.plugin [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] No Port match for: 68599998-f433-42bf-80ae-c64a075f3712 {{(pid=99041) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 31 12:12:17.812346 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron.plugins.ml2.rpc [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Device 68599998-f433-42bf-80ae-c64a075f3712 not bound to the agent host dv-cinder-791239-4-iscsi {{(pid=99041) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 31 12:12:18.292459 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41076) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:18.340296 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-ef438dab-f8c4-42aa-99ae-89c4c82420d0 admin admin] 10.222.0.48 "GET /v2.0/security-groups?tenant_id=34d7d5411875496f938fb5fb58f45223&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0466509 May 31 12:12:18.342740 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.wsgi [-] (99040) accepted ('10.222.0.48', 41080) {{(pid=99040) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 31 12:12:18.402325 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8c5cbe0a-af56-42e4-8eac-86e0b3b208e7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:18.427219 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8c5cbe0a-af56-42e4-8eac-86e0b3b208e7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:18.544684 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8c5cbe0a-af56-42e4-8eac-86e0b3b208e7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-412540'] for security_group, after_delete {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:18.548450 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-9a7fd613-cd64-430b-a3e6-cf7e96eac84f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 31 12:12:18.562623 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8c5cbe0a-af56-42e4-8eac-86e0b3b208e7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=524cc778-95f3-429b-802b-40668b0604e0,revision_number=None']} {{(pid=99040) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 31 12:12:18.565421 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG oslo_concurrency.lockutils [None req-8c5cbe0a-af56-42e4-8eac-86e0b3b208e7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99040) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 31 12:12:18.566371 dv-cinder-791239-4-iscsi neutron-server[99040]: DEBUG neutron_lib.callbacks.manager [None req-8c5cbe0a-af56-42e4-8eac-86e0b3b208e7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99040) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:18.567444 dv-cinder-791239-4-iscsi neutron-server[99040]: INFO neutron.wsgi [None req-8c5cbe0a-af56-42e4-8eac-86e0b3b208e7 admin admin] 10.222.0.48 "DELETE /v2.0/security-groups/524cc778-95f3-429b-802b-40668b0604e0 HTTP/1.1" status: 204 len: 168 time: 0.2238469 May 31 12:12:20.931447 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-21ae626b-85cf-4a63-842c-87b42f752e6d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:23.333163 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:23.439635 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-84588400-c469-4b04-98d5-6a9fd3ad7781 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:24.652812 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:26.167126 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:29.094520 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [req-7ce31542-cca8-4618-9617-f2898bef4076 req-2a944974-e18e-471e-902e-816da6fa64ed service neutron] (99039) timed out ('10.222.0.48', 33794) {{(pid=99039) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 31 12:12:32.452597 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:12:39.444649 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:50.912793 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ad865970-c8ab-4ab9-83b0-07e8cb02f192 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:53.358177 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:53.443847 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-779b8a53-7c0f-473d-8bb2-1ee349013675 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:54.666714 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:12:56.158182 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:01.120998 dv-cinder-791239-4-iscsi neutron-server[99039]: DEBUG neutron.wsgi [req-b9856db8-c2bf-4886-9b67-0a451349a58b req-0771c368-337d-4e43-b083-ea810dcd46f3 tempest-AttachVolumeNegativeTest-565682384 tempest-AttachVolumeNegativeTest-565682384-project] (99039) timed out ('10.222.0.47', 49580) {{(pid=99039) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 31 12:13:09.452988 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:13:09.460671 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:20.911524 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-0503838d-f70c-4bfe-9ed7-6f2ca7f26d11 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:23.337069 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:23.459689 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3908bd51-0126-4ceb-915f-9edd2334e423 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:24.665838 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:26.145099 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:39.478735 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:46.452231 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:13:50.921236 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f57a45fd-cf04-4eb6-b7f9-26d60f898e56 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:53.367461 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:53.487140 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-30e8231b-03d4-4c68-b7f9-a286279a575d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:54.669476 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:13:56.149617 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:09.463359 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:20.957709 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c94389eb-aa49-4c1a-bc84-81cb108eb7db None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:23.329123 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:23.437797 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-0e2e20e8-d31d-4f3e-9031-693999d0dd48 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:23.451866 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:14:24.672380 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:26.165555 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:39.473243 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:50.947604 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-79ab7fa5-53a1-4976-a1c8-468169a29c8a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:53.335894 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:53.461532 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-c2081d31-98bb-413e-a8ce-e6f53d4fb9f0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:54.672521 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:14:56.232695 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:00.451641 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:15:09.504992 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:20.968491 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-170d3d88-989a-4662-8692-4a4477c821d7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:23.345963 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:23.442744 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-56f6b2ed-f210-454c-b9bd-177e3c84db75 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:24.682458 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:26.168667 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:37.452777 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:15:39.539130 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:50.910597 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-e567eaaf-a339-4f48-b61e-c3d1b59da32a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:53.361460 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:53.464253 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-6b25e36a-1952-4165-a5cb-d8f225145d8d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:54.680702 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:15:56.173415 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:09.508819 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:14.451695 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:16:20.915511 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-ef324a08-f385-4d1c-aa6f-17fcefcac3db None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:23.348769 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:23.463373 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-dcaf048d-6620-4e2a-9a34-6da99b1cbedd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:24.674254 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:26.183358 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:39.525703 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:50.926475 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-76d548d6-a1cb-4127-8e3d-3966e32e17b5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:51.452682 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 31 12:16:53.348477 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:53.451446 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1981e1c1-b780-4b51-8ed2-977601061372 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:54.685761 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:16:56.199578 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:17:09.518924 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-5b2228b8-6a3d-48c7-b6d7-169cb3882134 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:17:20.923265 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-1dd3b4a1-85e8-4a5d-af66-4d2d15d53836 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:17:23.336545 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-f832a7ac-e462-4208-9455-472af0a5c6f6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:17:23.427050 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-12189a8e-bc36-4c0e-95e0-ac27cbe3c4ee None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:17:24.661323 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-3db13b8d-d05b-4898-a1b5-d395094f0c1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:17:26.176039 dv-cinder-791239-4-iscsi neutron-server[99041]: DEBUG neutron_lib.callbacks.manager [None req-24436296-95c3-4d01-b510-5d241846259b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8759697518318', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1544592'] for agent, after_update {{(pid=99041) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 31 12:17:28.452760 dv-cinder-791239-4-iscsi neutron-server[99042]: DEBUG neutron.db.agents_db [None req-adc92f7d-9f52-4dce-b1a0-51c342587244 None None] Agent healthcheck: found 6 active agents {{(pid=99042) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}}