File System Summary =================== WARN: Device /dev/loop0 (/snap/core18/1997) is 100% full, might be an issue WARN: Device /dev/loop1 (/snap/lxd/19647) is 100% full, might be an issue WARN: Device /dev/loop2 (/snap/snapd/11588) is 100% full, might be an issue Filesystem Size Used Avail Use% Mounted on udev 5.9G 0 5.9G 0% /dev tmpfs 1.2G 1.2M 1.2G 1% /run /dev/sda1 39G 4.8G 34G 13% / tmpfs 5.9G 0 5.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 5.9G 0 5.9G 0% /sys/fs/cgroup /dev/loop0 56M 56M 0 100% /snap/core18/1997 /dev/loop1 71M 71M 0 100% /snap/lxd/19647 /dev/loop2 33M 33M 0 100% /snap/snapd/11588 /dev/sda15 105M 7.8M 97M 8% /boot/efi tmpfs 1.2G 0 1.2G 0% /run/user/1000 tmpfs 512M 0 512M 0% /opt/stack/data/etcd Process Listing =============== ps axo user,ppid,pid,pcpu,pmem,vsz,rss,tty,stat,start,time,args --------------------------------------------------------------- USER PPID PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 0 1 1.9 0.1 169096 13172 ? Ss 10:26:41 00:00:30 /sbin/init root 0 2 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [kthreadd] root 2 3 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [rcu_gp] root 2 4 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [rcu_par_gp] root 2 6 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [kworker/0:0H-kblockd] root 2 8 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [mm_percpu_wq] root 2 9 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [ksoftirqd/0] root 2 10 0.0 0.0 0 0 ? I 10:26:41 00:00:00 [rcu_sched] root 2 11 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [migration/0] root 2 12 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [idle_inject/0] root 2 14 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [cpuhp/0] root 2 15 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [cpuhp/1] root 2 16 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [idle_inject/1] root 2 17 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [migration/1] root 2 18 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [ksoftirqd/1] root 2 20 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [kworker/1:0H-kblockd] root 2 21 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [cpuhp/2] root 2 22 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [idle_inject/2] root 2 23 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [migration/2] root 2 24 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [ksoftirqd/2] root 2 26 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [kworker/2:0H-kblockd] root 2 27 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [cpuhp/3] root 2 28 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [idle_inject/3] root 2 29 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [migration/3] root 2 30 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [ksoftirqd/3] root 2 32 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [kworker/3:0H-kblockd] root 2 33 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [kdevtmpfs] root 2 34 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [netns] root 2 35 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [rcu_tasks_kthre] root 2 36 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [kauditd] root 2 39 0.0 0.0 0 0 ? I 10:26:41 00:00:00 [kworker/1:2-rcu_par_gp] root 2 40 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [khungtaskd] root 2 41 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [oom_reaper] root 2 42 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [writeback] root 2 43 0.0 0.0 0 0 ? S 10:26:41 00:00:00 [kcompactd0] root 2 44 0.0 0.0 0 0 ? SN 10:26:41 00:00:00 [ksmd] root 2 45 0.0 0.0 0 0 ? SN 10:26:41 00:00:00 [khugepaged] root 2 91 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [kintegrityd] root 2 92 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [kblockd] root 2 93 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [blkcg_punt_bio] root 2 94 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [tpm_dev_wq] root 2 95 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [ata_sff] root 2 96 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [md] root 2 97 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [edac-poller] root 2 98 0.0 0.0 0 0 ? I< 10:26:41 00:00:00 [devfreq_wq] root 2 99 0.0 0.0 0 0 ? I 10:26:42 00:00:00 [kworker/2:1-events] root 2 100 0.0 0.0 0 0 ? S 10:26:42 00:00:00 [watchdogd] root 2 105 0.0 0.0 0 0 ? S 10:26:42 00:00:00 [kswapd0] root 2 106 0.0 0.0 0 0 ? S 10:26:42 00:00:00 [ecryptfs-kthrea] root 2 108 0.0 0.0 0 0 ? I< 10:26:42 00:00:00 [kthrotld] root 2 109 0.0 0.0 0 0 ? I< 10:26:42 00:00:00 [acpi_thermal_pm] root 2 110 0.0 0.0 0 0 ? I 10:26:42 00:00:00 [kworker/3:2-events] root 2 111 0.0 0.0 0 0 ? S 10:26:42 00:00:00 [scsi_eh_0] root 2 112 0.0 0.0 0 0 ? I< 10:26:42 00:00:00 [scsi_tmf_0] root 2 113 0.0 0.0 0 0 ? S 10:26:42 00:00:00 [scsi_eh_1] root 2 114 0.0 0.0 0 0 ? I< 10:26:42 00:00:00 [scsi_tmf_1] root 2 116 0.0 0.0 0 0 ? I< 10:26:42 00:00:00 [vfio-irqfd-clea] root 2 117 0.0 0.0 0 0 ? I< 10:26:43 00:00:00 [ipv6_addrconf] root 2 127 0.0 0.0 0 0 ? I< 10:26:43 00:00:00 [kstrp] root 2 130 0.0 0.0 0 0 ? I< 10:26:43 00:00:00 [kworker/u257:0] root 2 143 0.0 0.0 0 0 ? I< 10:26:43 00:00:00 [charger_manager] root 2 183 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [kworker/3:1H-kblockd] root 2 184 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [hv_vmbus_con] root 2 185 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [hv_pri_chan] root 2 186 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [hv_sub_chan] root 2 188 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [cryptd] root 2 236 0.0 0.0 0 0 ? S 10:26:44 00:00:00 [scsi_eh_2] root 2 237 0.0 0.0 0 0 ? S 10:26:44 00:00:00 [scsi_eh_3] root 2 238 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [scsi_tmf_2] root 2 239 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [scsi_tmf_3] root 2 240 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [storvsc_error_w] root 2 241 0.0 0.0 0 0 ? I< 10:26:44 00:00:00 [storvsc_error_w] root 2 242 0.0 0.0 0 0 ? I 10:26:44 00:00:00 [kworker/0:3-events] root 2 276 0.0 0.0 0 0 ? I< 10:26:46 00:00:00 [raid5wq] root 2 302 0.0 0.0 0 0 ? I< 10:26:46 00:00:00 [kworker/2:1H-kblockd] root 2 317 0.0 0.0 0 0 ? S 10:26:46 00:00:01 [jbd2/sda1-8] root 2 318 0.0 0.0 0 0 ? I< 10:26:46 00:00:00 [ext4-rsv-conver] root 2 350 0.0 0.0 0 0 ? I< 10:26:46 00:00:00 [kworker/0:1H-kblockd] root 2 357 0.0 0.0 0 0 ? I< 10:26:47 00:00:00 [kworker/1:1H-kblockd] root 1 423 0.0 0.0 19352 5952 ? Ss 10:26:48 00:00:01 /lib/systemd/systemd-udevd root 2 452 0.0 0.0 0 0 ? S 10:26:49 00:00:00 [hv_balloon] root 2 550 0.0 0.0 0 0 ? I< 10:26:49 00:00:00 [kaluad] root 2 551 0.0 0.0 0 0 ? I< 10:26:49 00:00:00 [kmpath_rdacd] root 2 552 0.0 0.0 0 0 ? I< 10:26:49 00:00:00 [kmpathd] root 2 553 0.0 0.0 0 0 ? I< 10:26:49 00:00:00 [kmpath_handlerd] root 1 554 0.0 0.1 280136 17940 ? SLsl 10:26:49 00:00:00 /sbin/multipathd -d -s root 2 565 0.0 0.0 0 0 ? S< 10:26:49 00:00:00 [loop0] root 2 568 0.0 0.0 0 0 ? S< 10:26:49 00:00:00 [loop1] root 2 570 0.0 0.0 0 0 ? S< 10:26:49 00:00:00 [loop2] systemd+ 1 593 0.0 0.0 90228 6100 ? Ssl 10:26:49 00:00:00 /lib/systemd/systemd-timesyncd systemd+ 1 659 0.0 0.0 26720 7912 ? Ss 10:26:53 00:00:00 /lib/systemd/systemd-networkd systemd+ 1 661 0.0 0.1 23892 12272 ? Ss 10:26:53 00:00:00 /lib/systemd/systemd-resolved root 1 779 0.0 0.0 241048 9472 ? Ssl 10:26:58 00:00:00 /usr/lib/accountsservice/accounts-daemon root 1 783 0.0 0.0 8536 2948 ? Ss 10:26:58 00:00:00 /usr/sbin/cron -f message+ 1 784 0.1 0.0 7796 4904 ? Ss 10:26:58 00:00:03 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only root 1 792 0.0 0.0 81828 3748 ? Ssl 10:26:58 00:00:00 /usr/sbin/irqbalance --foreground root 1 796 0.0 0.1 29264 17988 ? Ss 10:26:58 00:00:00 /usr/bin/python3 /usr/bin/networkd-dispatcher --run-startup-triggers root 1 801 0.1 0.3 1298740 37388 ? Ssl 10:26:58 00:00:02 /usr/lib/snapd/snapd root 1 802 0.1 0.0 16780 7640 ? Ss 10:26:58 00:00:01 /lib/systemd/systemd-logind daemon 1 805 0.0 0.0 3792 2188 ? Ss 10:26:58 00:00:00 /usr/sbin/atd -f root 1 854 0.0 0.0 7352 2292 ttyS0 Ss+ 10:26:58 00:00:00 /sbin/agetty -o -p -- \u --keep-baud 115200,38400,9600 ttyS0 vt220 root 1 868 0.0 0.0 5828 1784 tty1 Ss+ 10:26:58 00:00:00 /sbin/agetty -o -p -- \u --noclear tty1 linux root 1 879 0.0 0.1 108100 20920 ? Ssl 10:26:58 00:00:00 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal root 1 883 0.0 0.0 237364 10012 ? Ssl 10:26:58 00:00:00 /usr/lib/policykit-1/polkitd --no-debug root 1 910 0.0 0.0 12176 7328 ? Ss 10:26:59 00:00:00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups root 910 1580 0.0 0.0 13796 9044 ? Ss 10:30:22 00:00:00 sshd: ubuntu [priv] ubuntu 1 1594 0.0 0.0 18384 9520 ? Ss 10:30:22 00:00:00 /lib/systemd/systemd --user ubuntu 1594 1598 0.0 0.0 169868 4564 ? S 10:30:22 00:00:00 (sd-pam) ubuntu 1580 1673 0.0 0.0 13928 6568 ? S 10:30:23 00:00:00 sshd: ubuntu@pts/0 root 2 8030 0.0 0.0 0 0 ? I 10:32:20 00:00:00 [kworker/u256:0-events_power_efficient] root 2 8157 0.0 0.0 0 0 ? I 10:36:44 00:00:00 [kworker/0:0-rcu_par_gp] ubuntu 1673 8762 0.0 0.0 2608 544 pts/0 Ss+ 10:41:14 00:00:00 /bin/sh -c /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1622544075.32-154896798164983/AnsiballZ_command.py && sleep 0 ubuntu 8762 8763 0.0 0.1 27804 17852 pts/0 S+ 10:41:14 00:00:00 /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1622544075.32-154896798164983/AnsiballZ_command.py ubuntu 8763 8765 0.0 0.0 2608 612 pts/0 S+ 10:41:15 00:00:00 /bin/sh -c ./unstack.sh > /dev/null 2>&1 ./stack.sh > /dev/null 2>&1 root 2 9949 0.0 0.0 2488 588 ? S 10:41:17 00:00:00 bpfilter_umh root 2 11053 0.0 0.0 0 0 ? I 10:41:19 00:00:00 [kworker/2:2-events] root 2 11081 0.0 0.0 0 0 ? I 10:41:19 00:00:00 [kworker/u256:1-events_power_efficient] root 2 11082 0.0 0.0 0 0 ? I 10:41:19 00:00:00 [kworker/3:0-events] ubuntu 8765 11506 0.5 0.0 14764 10048 pts/0 S+ 10:41:21 00:00:03 bash ./stack.sh ubuntu 11506 13761 0.0 0.0 10680 3900 pts/0 S+ 10:41:24 00:00:00 bash ./stack.sh ubuntu 11506 13766 0.0 0.0 10680 3836 pts/0 S+ 10:41:24 00:00:00 bash ./stack.sh ubuntu 13761 13768 0.1 0.0 17368 11376 pts/0 S+ 10:41:24 00:00:00 /usr/bin/python3.8 /home/ubuntu/devstack/tools/outfilter.py -v --no-timestamp -o /opt/stack/logs/stack.sh.log.2021-06-01-104124 ubuntu 13766 13773 0.0 0.0 17368 11372 pts/0 S+ 10:41:24 00:00:00 /usr/bin/python3.8 /home/ubuntu/devstack/tools/outfilter.py -o /opt/stack/logs/stack.sh.log.2021-06-01-104124.summary.2021-06-01-104124 root 1 15462 0.0 0.1 283804 16072 ? Ssl 10:41:26 00:00:00 /usr/lib/packagekit/packagekitd root 2 15594 0.0 0.0 0 0 ? I 10:41:58 00:00:00 [kworker/1:1-mm_percpu_wq] root 2 15595 0.0 0.0 0 0 ? I 10:41:58 00:00:00 [kworker/1:3-events] memcache 1 18162 0.0 0.0 406260 2736 ? Ssl 10:42:39 00:00:00 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1 -P /var/run/memcached/memcached.pid root 1 18392 0.0 0.0 7744 2972 ? Ss 10:42:43 00:00:00 /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf root 1 18531 0.0 0.0 15304 9076 ? Ss 10:42:45 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock haproxy 18531 18532 0.0 0.0 236500 3004 ? Sl 10:42:45 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock pcp 1 20933 0.1 0.0 8356 4520 ? Ss 10:43:03 00:00:01 /usr/lib/pcp/bin/pmcd root 20933 20936 0.0 0.0 7736 4036 ? S 10:43:03 00:00:00 /var/lib/pcp/pmdas/root/pmdaroot root 20936 20937 0.0 0.0 8308 5012 ? S 10:43:03 00:00:00 /var/lib/pcp/pmdas/proc/pmdaproc -d 3 root 20936 20938 0.0 0.0 7212 1636 ? S 10:43:03 00:00:00 /var/lib/pcp/pmdas/xfs/pmdaxfs -d 11 root 20936 20939 0.0 0.0 8268 5084 ? S 10:43:03 00:00:00 /var/lib/pcp/pmdas/linux/pmdalinux root 20936 20940 0.0 0.0 7268 3744 ? S 10:43:03 00:00:00 /var/lib/pcp/pmdas/kvm/pmdakvm -d 95 pcp 1 31895 0.1 0.0 8428 5212 ? S 10:43:17 00:00:00 /usr/lib/pcp/bin/pmlogger -P -r -T24h10m -c config.default -v 100mb -m pmlogger_check 20210601.10.43 syslog 1 36787 0.0 0.0 224348 4316 ? Ssl 10:43:32 00:00:00 /usr/sbin/rsyslogd -n -iNONE root 1 45307 0.0 0.0 35028 10428 ? S mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:73:b3:b1 brd ff:ff:ff:ff:ff:ff inet 10.222.0.74/22 brd 10.222.3.255 scope global dynamic eth0 valid_lft 84859sec preferred_lft 84859sec inet6 fe80::f816:3eff:fe73:b3b1/64 scope link valid_lft forever preferred_lft forever 3: eth1: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:14:fa:38 brd ff:ff:ff:ff:ff:ff inet 192.168.0.66/22 brd 192.168.3.255 scope global dynamic eth1 valid_lft 84858sec preferred_lft 84858sec inet6 fe80::f816:3eff:fe14:fa38/64 scope link valid_lft forever preferred_lft forever ip route -------- default via 10.222.0.1 dev eth0 proto dhcp src 10.222.0.74 metric 100 10.222.0.0/22 dev eth0 proto kernel scope link src 10.222.0.74 10.222.0.0/22 dev eth1 proto dhcp scope link src 192.168.0.66 metric 100 169.254.169.254 via 192.168.0.12 dev eth1 proto dhcp src 192.168.0.66 metric 100 169.254.169.254 via 10.222.0.12 dev eth0 proto dhcp src 10.222.0.74 metric 100 ip -6 route ----------- ::1 dev lo proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium ip netns -------- Open vSwitch Dump ================= sudo ovs-vsctl show ------------------- 521c4cd7-7319-4fc3-ae4a-d6a5e0841680 ovs_version: "2.13.3" IP Tables Dump ============== sudo iptables --line-numbers -L -nv -t filter --------------------------------------------- Chain INPUT (policy ACCEPT 110K packets, 334M bytes) num pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 55825 packets, 3512K bytes) num pkts bytes target prot opt in out source destination sudo iptables --line-numbers -L -nv -t nat ------------------------------------------ Chain PREROUTING (policy ACCEPT 716 packets, 188K bytes) num pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 548 packets, 171K bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 545 packets, 47971 bytes) num pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 545 packets, 47971 bytes) num pkts bytes target prot opt in out source destination sudo iptables --line-numbers -L -nv -t mangle --------------------------------------------- Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination EB Tables Dump ============== sudo ebtables -t filter -L -------------------------- Bridge table: filter Bridge chain: INPUT, entries: 0, policy: ACCEPT Bridge chain: FORWARD, entries: 0, policy: ACCEPT Bridge chain: OUTPUT, entries: 0, policy: ACCEPT sudo ebtables -t nat -L ----------------------- Bridge table: nat Bridge chain: PREROUTING, entries: 0, policy: ACCEPT Bridge chain: OUTPUT, entries: 0, policy: ACCEPT Bridge chain: POSTROUTING, entries: 0, policy: ACCEPT sudo ebtables -t broute -L -------------------------- *** Failed to run 'sudo ebtables -t broute -L': Command 'sudo ebtables -t broute -L' returned non-zero exit status 1. Compute consoles ================ nova-compute Guru Meditation Report =================================== Skipping as nova-compute does not appear to be running neutron-dhcp-agent Guru Meditation Report ========================================= Skipping as neutron-dhcp-agent does not appear to be running neutron-l3-agent Guru Meditation Report ======================================= Skipping as neutron-l3-agent does not appear to be running neutron-linuxbridge-agent Guru Meditation Report ================================================ Skipping as neutron-linuxbridge-agent does not appear to be running neutron-metadata-agent Guru Meditation Report ============================================= Skipping as neutron-metadata-agent does not appear to be running neutron-openvswitch-agent Guru Meditation Report ================================================ Skipping as neutron-openvswitch-agent does not appear to be running cinder-volume Guru Meditation Report ==================================== Skipping as cinder-volume does not appear to be running