File System Summary =================== WARN: Device /dev/loop1 (/snap/snapd/11588) is 100% full, might be an issue WARN: Device /dev/loop0 (/snap/core18/1997) is 100% full, might be an issue WARN: Device /dev/loop2 (/snap/lxd/19647) is 100% full, might be an issue Filesystem Size Used Avail Use% Mounted on udev 5.9G 0 5.9G 0% /dev tmpfs 1.2G 1.2M 1.2G 1% /run /dev/sda1 39G 4.9G 34G 13% / tmpfs 5.9G 0 5.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 5.9G 0 5.9G 0% /sys/fs/cgroup /dev/loop1 33M 33M 0 100% /snap/snapd/11588 /dev/loop0 56M 56M 0 100% /snap/core18/1997 /dev/loop2 71M 71M 0 100% /snap/lxd/19647 /dev/sda15 105M 7.8M 97M 8% /boot/efi tmpfs 1.2G 0 1.2G 0% /run/user/1000 tmpfs 512M 0 512M 0% /opt/stack/data/etcd Process Listing =============== ps axo user,ppid,pid,pcpu,pmem,vsz,rss,tty,stat,start,time,args --------------------------------------------------------------- USER PPID PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 0 1 1.8 0.1 169012 13072 ? Ss 18:27:13 00:00:27 /sbin/init root 0 2 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [kthreadd] root 2 3 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [rcu_gp] root 2 4 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [rcu_par_gp] root 2 6 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [kworker/0:0H-kblockd] root 2 8 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [mm_percpu_wq] root 2 9 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [ksoftirqd/0] root 2 10 0.0 0.0 0 0 ? I 18:27:13 00:00:00 [rcu_sched] root 2 11 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [migration/0] root 2 12 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [idle_inject/0] root 2 14 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [cpuhp/0] root 2 15 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [cpuhp/1] root 2 16 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [idle_inject/1] root 2 17 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [migration/1] root 2 18 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [ksoftirqd/1] root 2 20 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [kworker/1:0H-kblockd] root 2 21 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [cpuhp/2] root 2 22 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [idle_inject/2] root 2 23 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [migration/2] root 2 24 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [ksoftirqd/2] root 2 26 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [kworker/2:0H-kblockd] root 2 27 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [cpuhp/3] root 2 28 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [idle_inject/3] root 2 29 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [migration/3] root 2 30 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [ksoftirqd/3] root 2 32 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [kworker/3:0H-kblockd] root 2 33 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [kdevtmpfs] root 2 34 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [netns] root 2 35 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [rcu_tasks_kthre] root 2 36 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [kauditd] root 2 37 0.0 0.0 0 0 ? I 18:27:13 00:00:00 [kworker/0:2-events] root 2 38 0.0 0.0 0 0 ? I 18:27:13 00:00:00 [kworker/1:1-events] root 2 40 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [khungtaskd] root 2 41 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [oom_reaper] root 2 42 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [writeback] root 2 43 0.0 0.0 0 0 ? S 18:27:13 00:00:00 [kcompactd0] root 2 44 0.0 0.0 0 0 ? SN 18:27:13 00:00:00 [ksmd] root 2 45 0.0 0.0 0 0 ? SN 18:27:13 00:00:00 [khugepaged] root 2 91 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [kintegrityd] root 2 92 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [kblockd] root 2 93 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [blkcg_punt_bio] root 2 94 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [tpm_dev_wq] root 2 95 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [ata_sff] root 2 96 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [md] root 2 97 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [edac-poller] root 2 98 0.0 0.0 0 0 ? I< 18:27:13 00:00:00 [devfreq_wq] root 2 100 0.0 0.0 0 0 ? S 18:27:14 00:00:00 [watchdogd] root 2 102 0.0 0.0 0 0 ? I 18:27:14 00:00:00 [kworker/3:1-rcu_par_gp] root 2 105 0.0 0.0 0 0 ? S 18:27:14 00:00:00 [kswapd0] root 2 106 0.0 0.0 0 0 ? S 18:27:14 00:00:00 [ecryptfs-kthrea] root 2 108 0.0 0.0 0 0 ? I< 18:27:14 00:00:00 [kthrotld] root 2 109 0.0 0.0 0 0 ? I< 18:27:14 00:00:00 [acpi_thermal_pm] root 2 110 0.0 0.0 0 0 ? S 18:27:14 00:00:00 [scsi_eh_0] root 2 111 0.0 0.0 0 0 ? I< 18:27:14 00:00:00 [scsi_tmf_0] root 2 112 0.0 0.0 0 0 ? S 18:27:14 00:00:00 [scsi_eh_1] root 2 113 0.0 0.0 0 0 ? I< 18:27:14 00:00:00 [scsi_tmf_1] root 2 115 0.0 0.0 0 0 ? I< 18:27:14 00:00:00 [vfio-irqfd-clea] root 2 116 0.0 0.0 0 0 ? I< 18:27:15 00:00:00 [ipv6_addrconf] root 2 126 0.0 0.0 0 0 ? I< 18:27:15 00:00:00 [kstrp] root 2 130 0.0 0.0 0 0 ? I< 18:27:15 00:00:00 [kworker/u257:0] root 2 143 0.0 0.0 0 0 ? I< 18:27:15 00:00:00 [charger_manager] root 2 181 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [hv_vmbus_con] root 2 182 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [hv_pri_chan] root 2 183 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [hv_sub_chan] root 2 189 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [kworker/3:1H-kblockd] root 2 190 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [cryptd] root 2 217 0.0 0.0 0 0 ? I 18:27:16 00:00:01 [kworker/u256:4-events_power_efficient] root 2 220 0.0 0.0 0 0 ? S 18:27:16 00:00:00 [scsi_eh_2] root 2 221 0.0 0.0 0 0 ? S 18:27:16 00:00:00 [scsi_eh_3] root 2 222 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [scsi_tmf_2] root 2 223 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [scsi_tmf_3] root 2 225 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [storvsc_error_w] root 2 226 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [storvsc_error_w] root 2 240 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [kworker/1:1H-kblockd] root 2 241 0.0 0.0 0 0 ? I< 18:27:16 00:00:00 [kworker/2:1H-kblockd] root 2 275 0.0 0.0 0 0 ? I< 18:27:18 00:00:00 [raid5wq] root 2 315 0.0 0.0 0 0 ? I< 18:27:18 00:00:00 [kworker/0:1H-kblockd] root 2 316 0.0 0.0 0 0 ? S 18:27:18 00:00:01 [jbd2/sda1-8] root 2 317 0.0 0.0 0 0 ? I< 18:27:18 00:00:00 [ext4-rsv-conver] root 1 420 0.0 0.0 19344 5800 ? Ss 18:27:20 00:00:00 /lib/systemd/systemd-udevd root 2 448 0.0 0.0 0 0 ? S 18:27:20 00:00:00 [hv_balloon] root 2 548 0.0 0.0 0 0 ? I< 18:27:21 00:00:00 [kaluad] root 2 549 0.0 0.0 0 0 ? I< 18:27:21 00:00:00 [kmpath_rdacd] root 2 550 0.0 0.0 0 0 ? I< 18:27:21 00:00:00 [kmpathd] root 2 551 0.0 0.0 0 0 ? I< 18:27:21 00:00:00 [kmpath_handlerd] root 1 552 0.0 0.1 280136 17940 ? SLsl 18:27:21 00:00:00 /sbin/multipathd -d -s root 2 561 0.0 0.0 0 0 ? S< 18:27:21 00:00:00 [loop0] root 2 565 0.0 0.0 0 0 ? S< 18:27:21 00:00:00 [loop1] root 2 568 0.0 0.0 0 0 ? S< 18:27:21 00:00:00 [loop2] systemd+ 1 592 0.0 0.0 90228 6072 ? Ssl 18:27:21 00:00:00 /lib/systemd/systemd-timesyncd systemd+ 1 658 0.0 0.0 26720 7984 ? Ss 18:27:24 00:00:00 /lib/systemd/systemd-networkd systemd+ 1 660 0.0 0.0 23892 12020 ? Ss 18:27:24 00:00:00 /lib/systemd/systemd-resolved root 1 780 0.0 0.0 241048 9436 ? Ssl 18:27:29 00:00:00 /usr/lib/accountsservice/accounts-daemon root 1 783 0.0 0.0 8536 2972 ? Ss 18:27:29 00:00:00 /usr/sbin/cron -f message+ 1 784 0.1 0.0 7776 4980 ? Ss 18:27:29 00:00:02 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only root 1 789 0.0 0.0 81828 3804 ? Ssl 18:27:29 00:00:00 /usr/sbin/irqbalance --foreground root 1 790 0.0 0.1 29264 17860 ? Ss 18:27:29 00:00:00 /usr/bin/python3 /usr/bin/networkd-dispatcher --run-startup-triggers root 1 794 0.1 0.2 1299800 35444 ? Ssl 18:27:29 00:00:02 /usr/lib/snapd/snapd root 1 798 0.1 0.0 16812 7788 ? Ss 18:27:29 00:00:01 /lib/systemd/systemd-logind daemon 1 805 0.0 0.0 3792 2288 ? Ss 18:27:29 00:00:00 /usr/sbin/atd -f root 1 857 0.0 0.0 7352 2252 ttyS0 Ss+ 18:27:30 00:00:00 /sbin/agetty -o -p -- \u --keep-baud 115200,38400,9600 ttyS0 vt220 root 1 866 0.0 0.1 108100 20680 ? Ssl 18:27:30 00:00:00 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal root 1 871 0.0 0.0 5828 1820 tty1 Ss+ 18:27:30 00:00:00 /sbin/agetty -o -p -- \u --noclear tty1 linux root 1 888 0.0 0.0 237364 10004 ? Ssl 18:27:30 00:00:00 /usr/lib/policykit-1/polkitd --no-debug root 1 919 0.0 0.0 12176 7560 ? Ss 18:27:31 00:00:00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups root 919 1575 0.0 0.0 13796 9144 ? Ss 18:32:19 00:00:00 sshd: ubuntu [priv] ubuntu 1 1591 0.0 0.0 18388 9548 ? Ss 18:32:19 00:00:00 /lib/systemd/systemd --user ubuntu 1591 1593 0.0 0.0 169788 4568 ? S 18:32:19 00:00:00 (sd-pam) ubuntu 1575 1672 0.0 0.0 13928 6624 ? S 18:32:20 00:00:00 sshd: ubuntu@pts/0 root 2 2738 0.0 0.0 0 0 ? I 18:33:09 00:00:00 [kworker/u256:0-events_power_efficient] root 2 2869 0.0 0.0 0 0 ? I 18:33:10 00:00:00 [kworker/1:0-events] root 2 8077 0.0 0.0 0 0 ? I 18:38:42 00:00:00 [kworker/2:0-cgroup_destroy] root 2 8192 0.0 0.0 0 0 ? I 18:42:31 00:00:00 [kworker/1:2-events] ubuntu 1672 8690 0.0 0.0 2608 596 pts/0 Ss+ 18:44:23 00:00:00 /bin/sh -c /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1620845064.19-157303783567782/AnsiballZ_command.py && sleep 0 ubuntu 8690 8691 0.0 0.1 27804 17960 pts/0 S+ 18:44:23 00:00:00 /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1620845064.19-157303783567782/AnsiballZ_command.py ubuntu 8691 8693 0.0 0.0 2608 612 pts/0 S+ 18:44:23 00:00:00 /bin/sh -c ./unstack.sh > /dev/null 2>&1 ./stack.sh > /dev/null 2>&1 root 2 9872 0.0 0.0 2488 588 ? S 18:44:25 00:00:00 bpfilter_umh root 2 9874 0.0 0.0 0 0 ? I 18:44:25 00:00:00 [kworker/3:0-events] root 2 11036 0.0 0.0 0 0 ? I 18:44:28 00:00:00 [kworker/2:2-rcu_par_gp] root 2 11038 0.0 0.0 0 0 ? I 18:44:28 00:00:00 [kworker/2:3-events] root 2 11066 0.0 0.0 0 0 ? I 18:44:28 00:00:00 [kworker/u256:1-events_power_efficient] ubuntu 8693 11477 0.6 0.0 15184 10452 pts/0 S+ 18:44:30 00:00:02 bash ./stack.sh ubuntu 11477 13744 0.0 0.0 10548 3744 pts/0 S+ 18:44:33 00:00:00 bash ./stack.sh ubuntu 11477 13749 0.0 0.0 10548 3744 pts/0 S+ 18:44:33 00:00:00 bash ./stack.sh ubuntu 13744 13750 0.1 0.0 17368 11256 pts/0 S+ 18:44:33 00:00:00 /usr/bin/python3.8 /home/ubuntu/devstack/tools/outfilter.py -v --no-timestamp -o /opt/stack/logs/stack.sh.log.2021-05-12-184434 ubuntu 13749 13755 0.0 0.0 17368 11328 pts/0 S+ 18:44:33 00:00:00 /usr/bin/python3.8 /home/ubuntu/devstack/tools/outfilter.py -o /opt/stack/logs/stack.sh.log.2021-05-12-184434.summary.2021-05-12-184434 root 1 15479 0.0 0.1 283804 15780 ? Ssl 18:44:36 00:00:00 /usr/lib/packagekit/packagekitd memcache 1 17992 0.0 0.0 406260 2956 ? Ssl 18:45:41 00:00:00 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1 -P /var/run/memcached/memcached.pid root 2 18607 0.0 0.0 0 0 ? I 18:45:48 00:00:00 [kworker/0:4-events] root 2 18608 0.0 0.0 0 0 ? I 18:45:48 00:00:00 [kworker/0:5-events] root 1 18609 0.0 0.0 7744 2964 ? Ss 18:45:48 00:00:00 /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf root 1 18748 0.0 0.0 15304 9004 ? Ss 18:45:50 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock haproxy 18748 18749 0.0 0.0 236500 2980 ? Sl 18:45:50 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock syslog 1 25014 0.0 0.0 224348 4332 ? Ssl 18:46:15 00:00:00 /usr/sbin/rsyslogd -n -iNONE root 1 32580 0.0 0.0 35028 10224 ? S mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:23:b9:bf brd ff:ff:ff:ff:ff:ff inet 10.222.0.44/22 brd 10.222.3.255 scope global dynamic eth0 valid_lft 84933sec preferred_lft 84933sec inet6 fe80::f816:3eff:fe23:b9bf/64 scope link valid_lft forever preferred_lft forever 3: eth1: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:21:19:6d brd ff:ff:ff:ff:ff:ff inet 192.168.0.47/22 brd 192.168.3.255 scope global dynamic eth1 valid_lft 84932sec preferred_lft 84932sec inet6 fe80::f816:3eff:fe21:196d/64 scope link valid_lft forever preferred_lft forever ip route -------- default via 10.222.0.1 dev eth0 proto dhcp src 10.222.0.44 metric 100 10.222.0.0/22 dev eth0 proto kernel scope link src 10.222.0.44 10.222.0.0/22 dev eth1 proto dhcp scope link src 192.168.0.47 metric 100 169.254.169.254 via 192.168.0.12 dev eth1 proto dhcp src 192.168.0.47 metric 100 169.254.169.254 via 10.222.0.12 dev eth0 proto dhcp src 10.222.0.44 metric 100 ip -6 route ----------- ::1 dev lo proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium ip netns -------- Open vSwitch Dump ================= sudo ovs-vsctl show ------------------- 043af465-09f8-475d-9f65-1ab6a58cb50d ovs_version: "2.13.1" IP Tables Dump ============== sudo iptables --line-numbers -L -nv -t filter --------------------------------------------- Chain INPUT (policy ACCEPT 86845 packets, 282M bytes) num pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 35530 packets, 2489K bytes) num pkts bytes target prot opt in out source destination sudo iptables --line-numbers -L -nv -t nat ------------------------------------------ Chain PREROUTING (policy ACCEPT 389 packets, 108K bytes) num pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 317 packets, 102K bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 537 packets, 42952 bytes) num pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 537 packets, 42952 bytes) num pkts bytes target prot opt in out source destination sudo iptables --line-numbers -L -nv -t mangle --------------------------------------------- Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination EB Tables Dump ============== sudo ebtables -t filter -L -------------------------- Bridge table: filter Bridge chain: INPUT, entries: 0, policy: ACCEPT Bridge chain: FORWARD, entries: 0, policy: ACCEPT Bridge chain: OUTPUT, entries: 0, policy: ACCEPT sudo ebtables -t nat -L ----------------------- Bridge table: nat Bridge chain: PREROUTING, entries: 0, policy: ACCEPT Bridge chain: OUTPUT, entries: 0, policy: ACCEPT Bridge chain: POSTROUTING, entries: 0, policy: ACCEPT sudo ebtables -t broute -L -------------------------- *** Failed to run 'sudo ebtables -t broute -L': Command 'sudo ebtables -t broute -L' returned non-zero exit status 1. Compute consoles ================ nova-compute Guru Meditation Report =================================== Skipping as nova-compute does not appear to be running neutron-dhcp-agent Guru Meditation Report ========================================= Skipping as neutron-dhcp-agent does not appear to be running neutron-l3-agent Guru Meditation Report ======================================= Skipping as neutron-l3-agent does not appear to be running neutron-linuxbridge-agent Guru Meditation Report ================================================ Skipping as neutron-linuxbridge-agent does not appear to be running neutron-metadata-agent Guru Meditation Report ============================================= Skipping as neutron-metadata-agent does not appear to be running neutron-openvswitch-agent Guru Meditation Report ================================================ Skipping as neutron-openvswitch-agent does not appear to be running cinder-volume Guru Meditation Report ==================================== Skipping as cinder-volume does not appear to be running