File System Summary =================== WARN: Device /dev/loop1 (/snap/lxd/19647) is 100% full, might be an issue WARN: Device /dev/loop0 (/snap/core18/1997) is 100% full, might be an issue WARN: Device /dev/loop2 (/snap/snapd/11588) is 100% full, might be an issue Filesystem Size Used Avail Use% Mounted on udev 5.9G 0 5.9G 0% /dev tmpfs 1.2G 1.2M 1.2G 1% /run /dev/sda1 39G 3.5G 36G 10% / tmpfs 5.9G 0 5.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 5.9G 0 5.9G 0% /sys/fs/cgroup /dev/loop1 71M 71M 0 100% /snap/lxd/19647 /dev/loop0 56M 56M 0 100% /snap/core18/1997 /dev/loop2 33M 33M 0 100% /snap/snapd/11588 /dev/sda15 105M 7.8M 97M 8% /boot/efi tmpfs 1.2G 0 1.2G 0% /run/user/1000 Process Listing =============== ps axo user,ppid,pid,pcpu,pmem,vsz,rss,tty,stat,start,time,args --------------------------------------------------------------- USER PPID PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 0 1 1.7 0.1 168728 12940 ? Ss 11:58:34 00:00:23 /lib/systemd/systemd --system --deserialize 31 root 0 2 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [kthreadd] root 2 3 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [rcu_gp] root 2 4 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [rcu_par_gp] root 2 6 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [kworker/0:0H-kblockd] root 2 8 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [mm_percpu_wq] root 2 9 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [ksoftirqd/0] root 2 10 0.0 0.0 0 0 ? I 11:58:34 00:00:00 [rcu_sched] root 2 11 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [migration/0] root 2 12 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [idle_inject/0] root 2 14 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [cpuhp/0] root 2 15 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [cpuhp/1] root 2 16 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [idle_inject/1] root 2 17 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [migration/1] root 2 18 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [ksoftirqd/1] root 2 20 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [kworker/1:0H] root 2 21 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [cpuhp/2] root 2 22 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [idle_inject/2] root 2 23 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [migration/2] root 2 24 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [ksoftirqd/2] root 2 26 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [kworker/2:0H-kblockd] root 2 27 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [cpuhp/3] root 2 28 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [idle_inject/3] root 2 29 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [migration/3] root 2 30 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [ksoftirqd/3] root 2 32 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [kworker/3:0H-kblockd] root 2 33 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [kdevtmpfs] root 2 34 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [netns] root 2 35 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [rcu_tasks_kthre] root 2 36 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [kauditd] root 2 38 0.0 0.0 0 0 ? I 11:58:34 00:00:00 [kworker/1:2-events] root 2 41 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [khungtaskd] root 2 42 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [oom_reaper] root 2 43 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [writeback] root 2 44 0.0 0.0 0 0 ? S 11:58:34 00:00:00 [kcompactd0] root 2 45 0.0 0.0 0 0 ? SN 11:58:34 00:00:00 [ksmd] root 2 46 0.0 0.0 0 0 ? SN 11:58:34 00:00:00 [khugepaged] root 2 93 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [kintegrityd] root 2 94 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [kblockd] root 2 95 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [blkcg_punt_bio] root 2 96 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [tpm_dev_wq] root 2 97 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [ata_sff] root 2 98 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [md] root 2 99 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [edac-poller] root 2 100 0.0 0.0 0 0 ? I< 11:58:34 00:00:00 [devfreq_wq] root 2 101 0.0 0.0 0 0 ? S 11:58:35 00:00:00 [watchdogd] root 2 105 0.0 0.0 0 0 ? S 11:58:35 00:00:00 [kswapd0] root 2 106 0.0 0.0 0 0 ? S 11:58:35 00:00:00 [ecryptfs-kthrea] root 2 108 0.0 0.0 0 0 ? I< 11:58:35 00:00:00 [kthrotld] root 2 109 0.0 0.0 0 0 ? I< 11:58:35 00:00:00 [acpi_thermal_pm] root 2 110 0.0 0.0 0 0 ? S 11:58:35 00:00:00 [scsi_eh_0] root 2 111 0.0 0.0 0 0 ? I< 11:58:35 00:00:00 [scsi_tmf_0] root 2 112 0.0 0.0 0 0 ? S 11:58:35 00:00:00 [scsi_eh_1] root 2 113 0.0 0.0 0 0 ? I< 11:58:35 00:00:00 [scsi_tmf_1] root 2 115 0.0 0.0 0 0 ? I< 11:58:35 00:00:00 [vfio-irqfd-clea] root 2 116 0.0 0.0 0 0 ? I< 11:58:36 00:00:00 [ipv6_addrconf] root 2 117 0.0 0.0 0 0 ? I 11:58:36 00:00:00 [kworker/u256:3-events_power_efficient] root 2 126 0.0 0.0 0 0 ? I< 11:58:36 00:00:00 [kstrp] root 2 129 0.0 0.0 0 0 ? I< 11:58:36 00:00:00 [kworker/u257:0] root 2 142 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [charger_manager] root 2 183 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [hv_vmbus_con] root 2 186 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [hv_pri_chan] root 2 187 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [hv_sub_chan] root 2 188 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [kworker/3:1H-kblockd] root 2 189 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [cryptd] root 2 230 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [kworker/1:1H-kblockd] root 2 234 0.0 0.0 0 0 ? S 11:58:37 00:00:00 [scsi_eh_2] root 2 235 0.0 0.0 0 0 ? S 11:58:37 00:00:00 [scsi_eh_3] root 2 236 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [scsi_tmf_2] root 2 237 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [scsi_tmf_3] root 2 239 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [storvsc_error_w] root 2 240 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [storvsc_error_w] root 2 244 0.0 0.0 0 0 ? I< 11:58:37 00:00:00 [kworker/2:1H-kblockd] root 2 277 0.0 0.0 0 0 ? I< 11:58:39 00:00:00 [raid5wq] root 2 317 0.0 0.0 0 0 ? S 11:58:39 00:00:01 [jbd2/sda1-8] root 2 318 0.0 0.0 0 0 ? I< 11:58:39 00:00:00 [ext4-rsv-conver] root 2 354 0.0 0.0 0 0 ? I< 11:58:40 00:00:00 [kworker/0:1H-kblockd] root 1 423 0.0 0.0 19356 5864 ? Ss 11:58:42 00:00:00 /lib/systemd/systemd-udevd root 2 455 0.0 0.0 0 0 ? S 11:58:42 00:00:00 [hv_balloon] root 2 550 0.0 0.0 0 0 ? I< 11:58:42 00:00:00 [kaluad] root 2 551 0.0 0.0 0 0 ? I< 11:58:42 00:00:00 [kmpath_rdacd] root 2 552 0.0 0.0 0 0 ? I< 11:58:42 00:00:00 [kmpathd] root 2 553 0.0 0.0 0 0 ? I< 11:58:42 00:00:00 [kmpath_handlerd] root 1 554 0.0 0.1 280136 17940 ? SLsl 11:58:42 00:00:00 /sbin/multipathd -d -s root 2 562 0.0 0.0 0 0 ? S< 11:58:43 00:00:00 [loop0] root 2 566 0.0 0.0 0 0 ? S< 11:58:43 00:00:00 [loop1] root 2 568 0.0 0.0 0 0 ? S< 11:58:43 00:00:00 [loop2] root 1 770 0.0 0.0 241048 9508 ? Ssl 11:58:55 00:00:00 /usr/lib/accountsservice/accounts-daemon root 1 773 0.0 0.0 8536 3016 ? Ss 11:58:55 00:00:00 /usr/sbin/cron -f message+ 1 777 0.1 0.0 7776 4952 ? Ss 11:58:55 00:00:02 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only root 1 785 0.0 0.0 81828 3808 ? Ssl 11:58:55 00:00:00 /usr/sbin/irqbalance --foreground root 1 786 0.0 0.1 29264 17916 ? Ss 11:58:55 00:00:00 /usr/bin/python3 /usr/bin/networkd-dispatcher --run-startup-triggers root 1 791 0.2 0.2 1296628 34688 ? Ssl 11:58:55 00:00:02 /usr/lib/snapd/snapd root 1 792 0.0 0.0 16780 7784 ? Ss 11:58:55 00:00:01 /lib/systemd/systemd-logind daemon 1 794 0.0 0.0 3792 2120 ? Ss 11:58:55 00:00:00 /usr/sbin/atd -f root 1 825 0.0 0.0 7352 2220 ttyS0 Ss+ 11:58:55 00:00:00 /sbin/agetty -o -p -- \u --keep-baud 115200,38400,9600 ttyS0 vt220 root 1 836 0.0 0.0 5828 1880 tty1 Ss+ 11:58:55 00:00:00 /sbin/agetty -o -p -- \u --noclear tty1 linux root 2 857 0.0 0.0 0 0 ? I 11:58:55 00:00:00 [kworker/3:4-rcu_par_gp] root 1 861 0.0 0.1 108100 21072 ? Ssl 11:58:55 00:00:00 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal root 1 868 0.0 0.0 237748 10368 ? Ssl 11:58:55 00:00:00 /usr/lib/policykit-1/polkitd --no-debug root 1 1563 0.0 0.0 13800 8932 ? Ss 12:02:55 00:00:00 sshd: ubuntu [priv] ubuntu 1 1576 0.0 0.0 18384 9784 ? Ss 12:02:55 00:00:00 /lib/systemd/systemd --user ubuntu 1576 1581 0.0 0.0 169780 4564 ? S 12:02:55 00:00:00 (sd-pam) ubuntu 1563 1660 0.0 0.0 13936 6604 ? S 12:02:56 00:00:00 sshd: ubuntu@pts/0 root 2 2928 0.0 0.0 0 0 ? I 12:04:31 00:00:00 [kworker/u256:2-events_power_efficient] root 2 3131 0.0 0.0 0 0 ? I 12:04:33 00:00:00 [kworker/2:0-rcu_par_gp] root 2 13710 0.0 0.0 0 0 ? I 12:05:37 00:00:00 [kworker/1:0-events] root 2 14291 0.0 0.0 0 0 ? I 12:10:46 00:00:00 [kworker/0:1-events] root 2 14306 0.0 0.0 0 0 ? I 12:10:46 00:00:00 [kworker/0:2-memcg_kmem_cache] root 2 14524 0.0 0.0 0 0 ? I 12:13:40 00:00:00 [kworker/3:1-events] root 2 14527 0.0 0.0 0 0 ? I 12:13:40 00:00:00 [kworker/3:2-events] ubuntu 1660 14785 0.0 0.0 2608 608 pts/0 Ss+ 12:15:01 00:00:00 /bin/sh -c /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1640693701.25-251034909820581/AnsiballZ_command.py && sleep 0 ubuntu 14785 14786 0.0 0.1 28088 17928 pts/0 S+ 12:15:01 00:00:00 /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1640693701.25-251034909820581/AnsiballZ_command.py ubuntu 14786 14788 0.0 0.0 2608 608 pts/0 S+ 12:15:01 00:00:00 /bin/sh -c ./unstack.sh > /dev/null 2>&1 ./stack.sh > /dev/null 2>&1 root 2 16003 0.0 0.0 2488 592 ? S 12:15:03 00:00:00 bpfilter_umh root 2 16028 0.0 0.0 0 0 ? I 12:15:03 00:00:00 [kworker/2:1-mm_percpu_wq] root 2 17128 0.0 0.0 0 0 ? I 12:15:05 00:00:00 [kworker/1:1-events] root 2 17169 0.0 0.0 0 0 ? I 12:15:05 00:00:00 [kworker/u256:1-events_power_efficient] root 2 17179 0.0 0.0 0 0 ? I 12:15:05 00:00:00 [kworker/u256:4-events_power_efficient] ubuntu 14788 17537 0.1 0.0 14908 10132 pts/0 S+ 12:15:07 00:00:00 bash ./stack.sh ubuntu 17537 19778 0.0 0.0 10644 3892 pts/0 S+ 12:15:10 00:00:00 bash ./stack.sh ubuntu 17537 19783 0.0 0.0 10644 3892 pts/0 S+ 12:15:10 00:00:00 bash ./stack.sh ubuntu 19778 19784 0.0 0.0 17384 11320 pts/0 S+ 12:15:10 00:00:00 /usr/bin/python3.8 /home/ubuntu/devstack/tools/outfilter.py -v --no-timestamp -o /opt/stack/logs/stack.sh.log.2021-12-28-121510 ubuntu 19783 19789 0.0 0.0 17384 11196 pts/0 S+ 12:15:10 00:00:00 /usr/bin/python3.8 /home/ubuntu/devstack/tools/outfilter.py -o /opt/stack/logs/stack.sh.log.2021-12-28-121510.summary.2021-12-28-121510 memcache 1 24576 0.0 0.0 409768 3980 ? Ssl 12:18:11 00:00:00 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1 -P /var/run/memcached/memcached.pid root 2 24577 0.0 0.0 0 0 ? I 12:18:11 00:00:00 [kworker/0:0-cgroup_destroy] root 2 24747 0.0 0.0 0 0 ? I 12:18:18 00:00:00 [kworker/0:3-events] root 1 24813 0.0 0.0 7744 2968 ? Ss 12:18:26 00:00:00 /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf root 1 24984 0.0 0.0 15304 9056 ? Ss 12:18:35 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock haproxy 24984 24987 0.0 0.0 236500 3072 ? Sl 12:18:35 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock root 1 26496 0.0 0.0 12176 7132 ? Ss 12:18:54 00:00:00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups root 1 26963 0.0 0.0 6532 4684 ? Ss 12:19:12 00:00:00 /usr/sbin/apache2 -k start www-data 26963 26965 0.0 0.0 1932316 4540 ? Sl 12:19:12 00:00:00 /usr/sbin/apache2 -k start www-data 26963 26966 0.0 0.0 1932316 4540 ? Sl 12:19:12 00:00:00 /usr/sbin/apache2 -k start pcp 1 27629 1.4 0.0 8388 4360 ? Ss 12:19:21 00:00:01 /usr/lib/pcp/bin/pmcd root 27629 27632 0.0 0.0 7736 3928 ? S 12:19:21 00:00:00 /var/lib/pcp/pmdas/root/pmdaroot root 27632 27633 0.0 0.0 8308 4932 ? S 12:19:21 00:00:00 /var/lib/pcp/pmdas/proc/pmdaproc -d 3 root 27632 27634 0.0 0.0 7212 1696 ? S 12:19:21 00:00:00 /var/lib/pcp/pmdas/xfs/pmdaxfs -d 11 root 27632 27635 0.4 0.0 8080 4792 ? S 12:19:21 00:00:00 /var/lib/pcp/pmdas/linux/pmdalinux root 27632 27636 0.0 0.0 7272 3820 ? S 12:19:21 00:00:00 /var/lib/pcp/pmdas/kvm/pmdakvm -d 95 pcp 1 38625 1.5 0.0 8280 4776 ? S 12:19:35 00:00:00 /usr/lib/pcp/bin/pmlogger -P -r -T24h10m -c config.default -v 100mb -m pmlogger_check 20211228.12.19 systemd+ 1 38864 0.1 0.0 34932 8116 ? Ss 12:19:38 00:00:00 /lib/systemd/systemd-networkd root 1 38865 0.0 0.0 9500 4868 ? S mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:40:74:45 brd ff:ff:ff:ff:ff:ff inet 10.222.0.16/22 brd 10.222.3.255 scope global dynamic eth0 valid_lft 86344sec preferred_lft 86344sec inet6 fe80::f816:3eff:fe40:7445/64 scope link valid_lft forever preferred_lft forever 3: eth1: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:eb:01:46 brd ff:ff:ff:ff:ff:ff inet 192.168.0.41/22 brd 192.168.3.255 scope global dynamic eth1 valid_lft 86344sec preferred_lft 86344sec inet6 fe80::f816:3eff:feeb:146/64 scope link valid_lft forever preferred_lft forever ip route -------- default via 10.222.0.1 dev eth0 proto dhcp src 10.222.0.16 metric 100 10.222.0.0/22 dev eth0 proto kernel scope link src 10.222.0.16 10.222.0.0/22 dev eth1 proto dhcp scope link src 192.168.0.41 metric 100 169.254.169.254 via 192.168.0.12 dev eth1 proto dhcp src 192.168.0.41 metric 100 169.254.169.254 via 10.222.0.12 dev eth0 proto dhcp src 10.222.0.16 metric 100 192.168.0.0/22 dev eth1 proto kernel scope link src 192.168.0.41 192.168.0.0/22 dev eth0 proto dhcp scope link src 10.222.0.16 metric 100 ip -6 route ----------- ::1 dev lo proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium ip netns -------- Open vSwitch Dump ================= sudo ovs-vsctl show ------------------- a7d8aed3-81bb-43af-9d60-3e8d122f173e ovs_version: "2.13.3" IP Tables Dump ============== sudo iptables --line-numbers -L -nv -t filter --------------------------------------------- Chain INPUT (policy ACCEPT 65135 packets, 170M bytes) num pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 19881 packets, 1509K bytes) num pkts bytes target prot opt in out source destination sudo iptables --line-numbers -L -nv -t nat ------------------------------------------ Chain PREROUTING (policy ACCEPT 381 packets, 103K bytes) num pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 285 packets, 91392 bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 429 packets, 38928 bytes) num pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 429 packets, 38928 bytes) num pkts bytes target prot opt in out source destination sudo iptables --line-numbers -L -nv -t mangle --------------------------------------------- Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination EB Tables Dump ============== sudo ebtables -t filter -L -------------------------- Bridge table: filter Bridge chain: INPUT, entries: 0, policy: ACCEPT Bridge chain: FORWARD, entries: 0, policy: ACCEPT Bridge chain: OUTPUT, entries: 0, policy: ACCEPT sudo ebtables -t nat -L ----------------------- Bridge table: nat Bridge chain: PREROUTING, entries: 0, policy: ACCEPT Bridge chain: OUTPUT, entries: 0, policy: ACCEPT Bridge chain: POSTROUTING, entries: 0, policy: ACCEPT Compute consoles ================ nova-compute Guru Meditation Report =================================== Skipping as nova-compute does not appear to be running neutron-dhcp-agent Guru Meditation Report ========================================= Skipping as neutron-dhcp-agent does not appear to be running neutron-l3-agent Guru Meditation Report ======================================= Skipping as neutron-l3-agent does not appear to be running neutron-linuxbridge-agent Guru Meditation Report ================================================ Skipping as neutron-linuxbridge-agent does not appear to be running neutron-metadata-agent Guru Meditation Report ============================================= Skipping as neutron-metadata-agent does not appear to be running neutron-openvswitch-agent Guru Meditation Report ================================================ Skipping as neutron-openvswitch-agent does not appear to be running cinder-volume Guru Meditation Report ==================================== Skipping as cinder-volume does not appear to be running