-
Story
-
Resolution: Won't Do
-
Minor
-
RH442 - RHEL 7 2 20150420
-
None
-
en-US (English)
URL:
Reporter RHNID:
Section: -
Language: en-US (English)|
Workaround:
Description: Start with xsos – a tool for sysadmins and support techs for description, screenshots, install info. Disclaimer: Yes, I wrote it.
I'm going through the RH442 material for the first time this week (via ILT courtesy of Victor Costea) and it strikes me that the introduction and lecture-delivery of quite a few topics could be enhanced by the visual aides that xsos could provide. Examples follow.
lspci – Chapter 5
The lspci output on the classroom VMs isn't all that interesting, so the usefulness of lspci might not be apparent until you look at a machine like this, where xsos parses out and humanizes the details about some of the most important things on the PCI bus.
# xsos --lspci
LSPCI
Net:
3 quad-port (12) NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter (rev 42)
2 dual-port (4) Emulex Corporation OneConnect 10Gb NIC (be3) (rev 01)
Storage:
2 dual-port (4) QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
(1) Hewlett-Packard Company Smart Array G6 controllers (rev 01)
2 dual-port (4) Emulex Corporation OneConnect 10Gb FCoE Initiator (be3) (rev 01)
VGA:
ATI Technologies Inc ES1000 (rev 02)
dmidecode, lscpu – Chapter 5
# xsos --bios --cpu
DMIDECODE
BIOS:
Vend: HP
Vers: P65
Date: 10/01/2013
BIOS Rev:
FW Rev: 1.80
System:
Mfr: HP
Prod: ProLiant DL580 G7
Vers: Not Specified
Ser: SGH406C3YH
UUID: 30333436-3638-4753-4834-303643335948
CPU:
4 of 4 CPU sockets populated, 10 cores/20 threads per CPU
40 total cores, 80 total threads
Mfr: Intel
Fam: Xeon
Freq: 2400 MHz
Vers: Intel(R) Xeon(R) CPU E7- 4870 @ 2.40GHz
Memory:
524288 MB (512 GB) total
32 of 64 DIMMs populated (system max capacity 512 GB)
CPU
80 logical processors (40 CPU cores)
4 Intel Xeon CPU E7- 4870 @ 2.40GHz (flags: aes,constant_tsc,ht,lm,pae,vmx)
└─20 threads / 10 cores each
Clearly, this can make it quickly apparent how useful the dmidecode can be – i.e., show you what info you can really get from it. (The BIOS output is created by parsing output from a single dmidecode command and the CPU output is created solely by parsing /proc/cpuinfo.)
Memory – Chapters 9, 12
# xsos --mem
MEMORY
Stats graphed as percent of MemTotal:
MemUsed ▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊ 99.4%
Buffers ▊................................................. 2.5%
Cached ▊................................................. 1.9%
HugePages ▊▊▊▊▊▊▊▊▊▊........................................ 19.8%
Dirty ▊▊▊▊.............................................. 7.4%
RAM:
504.8 GiB total ram
501.8 GiB (99%) used
479.6 GiB (95%) used excluding Buffers/Cached
37.33 GiB (7%) dirty
HugePages:
100 GiB pre-allocated to HugePages (20% of total ram)
75 GiB of HugePages (75%) in-use by applications
LowMem/Slab/PageTables/Shmem:
8.2 GiB (2%) of total ram used for Slab
0.01 GiB (0%) of total ram used for PageTables
0 GiB (0%) of total ram used for Shmem
Swap:
7.8 GiB (53%) used of 14.8 GiB total
This output is created solely from parsing /proc/meminfo. I think it does a decent job of helping you to visualize the most important things. You can also change the units used, e.g.:
# xsos --mem --unit m
MEMORY
Stats graphed as percent of MemTotal:
MemUsed ▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊ 99.4%
Buffers ▊................................................. 2.5%
Cached ▊................................................. 1.9%
HugePages ▊▊▊▊▊▊▊▊▊▊........................................ 19.8%
Dirty ▊▊▊▊.............................................. 7.4%
RAM:
516894 MiB total ram
513836 MiB (99.4%) used
491065 MiB (95%) used excluding Buffers/Cached
38223 MiB (7.4%) dirty
HugePages:
102400 MiB pre-allocated to HugePages (19.8% of total ram)
76800 MiB of HugePages (75%) in-use by applications
LowMem/Slab/PageTables/Shmem:
8400 MiB (1.6%) of total ram used for Slab
9 MiB (0%) of total ram used for PageTables
0 MiB (0%) of total ram used for Shmem
Swap:
7985 MiB (52.9%) used of 15108 MiB total
There is also a --ps option, run here with minimal verbosity.
# xsos --ps -v0
PS CHECK
Total number of processes:
3379
Top users of CPU & MEM:
USER %CPU %MEM RSS
root 21.6% 0.0% 0.18 GiB
xbarusr01 5.4% 2472.6% 789.60 GiB
17183 3.7% 0.0% 0.01 GiB
Uninteruptible sleep & Defunct processes:
USER PID %CPU %MEM VSZ-MiB RSS-MiB TTY STAT START TIME COMMAND
root 10617 0.0 0.0 4 0 pts/0 D+ 15:11 0:00 prelink
Top CPU-using processes:
USER PID %CPU %MEM VSZ-MiB RSS-MiB TTY STAT START TIME COMMAND
root 7053 16.7 0.0 114 7 pts/0 Sl+ 15:08 0:32 python
17183 10462 3.7 0.0 7 4 pts/1 S+ 15:11 0:00 top
root 10585 3.2 0.0 13 6 pts/0 S+ 15:11 0:00 /usr/lib/rpm/rpmv
xbarusr51 19335 1.6 1.1 539 369 ? SNl Dec05 72:04 /usr/java/jdk1.6.0_17/bin/java
root 345 1.6 0.0 0 0 ? S< Dec03 124:39 [kswapd0]
Top MEM-using processes:
USER PID %CPU %MEM VSZ-MiB RSS-MiB TTY STAT START TIME COMMAND
xbarusr51 30017 0.0 27.4 8 8909 ? S Dec03 0:00 /usr/local/apache/2.0.63/bin/httpd
xbarusr51 30016 0.0 27.4 8 8909 ? S Dec03 0:00 /usr/local/apache/2.0.63/bin/httpd
xbarusr51 30014 0.0 27.4 8 8909 ? S Dec03 0:00 /usr/local/apache/2.0.63/bin/httpd
xbarusr51 30005 0.0 27.4 8 8909 ? S Dec03 0:00 /usr/local/apache/2.0.63/bin/httpd
xbarusr51 30004 0.0 27.4 8 8909 ? S Dec03 0:00 /usr/local/apache/2.0.63/bin/httpd
(This example is from a system with tons of java apps using shared memory.)
ethtool – Chapter 11
# xsos --ethtool
ETHTOOL
Interface Status:
eth0 0000:03:00.0 link=up 100Mb/s full (autoneg=N) rx ring 200/2047 drv tg3 v3.122 / fw 5719-v1.34 NCSI v1.2.19.0
eth1 0000:03:00.1 link=up 100Mb/s full (autoneg=N) rx ring 200/2047 drv tg3 v3.122 / fw 5719-v1.34 NCSI v1.2.19.0
eth3 0000:03:00.2 link=DOWN rx ring 200/2047 drv tg3 v3.122 / fw 5719-v1.34 NCSI v1.2.19.0
eth3 0000:03:00.3 link=DOWN rx ring 200/2047 drv tg3 v3.122 / fw 5719-v1.34 NCSI v1.2.19.0
eth4 0000:0a:00.0 link=DOWN rx ring 200/2047 drv tg3 v3.122 / fw 5719-v1.34 NCSI v1.2.19.0
eth5 0000:0a:00.1 link=DOWN rx ring 200/2047 drv tg3 v3.122 / fw 5719-v1.34 NCSI v1.2.19.0
eth6 0000:0a:00.2 link=DOWN rx ring 200/2047 drv tg3 v3.122 / fw 5719-v1.34 NCSI v1.2.19.0
eth7 0000:0a:00.3 link=DOWN rx ring 200/2047 drv tg3 v3.122 / fw 5719-v1.34 NCSI v1.2.19.0
Interface Errors:
eth0 rx_align_errors: 60
rx_xoff_pause_rcvd: 4
rx_in_length_errors: 18446612167329402880
rx_out_length_errors: 130353
This option makes it easy to see names, PCI IDs, link status, ring buffers, and driver/fw version info for all NICs at a glance, all thanks to multiple ethtool commands run against each netdev.
sysctls - Chapters 3, 9, 11, 12
# xsos -s
SYSCTLS
kernel.
hostname = "jiop.usersys.redhat.com"
osrelease = "3.10.0-229.4.2.el7.x86_64"
tainted = "0" (kernel untainted)
random.boot_id = "78a54f6b-34d4-409c-b3e8-43f16603998a"
random.entropy_avail [bits] = "3077"
hung_task_panic [bool] = "0"
hung_task_timeout_secs = "120" (secs task must be D-state to trigger)
msgmax [bytes] = "8192"
msgmnb [bytes] = "16384"
msgmni [msg queues] = "31606"
panic [secs] = "0" (no autoreboot on panic)
panic_on_oops [bool] = "1"
nmi_watchdog [bool] = "1"
panic_on_io_nmi [bool] = "0"
panic_on_unrecovered_nmi [bool] = "0"
unknown_nmi_panic [bool] = "0"
panic_on_stackoverflow [bool] = "0"
softlockup_panic [bool] = "0"
softlockup_thresh [secs] = {sysctl not present}
pid_max = "32768"
threads-max = "126270"
sem [array] = "250 32000 32 128"
SEMMSL (max semaphores per array) = 250
SEMMNS (max sems system-wide) = 32000
SEMOPM (max ops per semop call) = 32
SEMMNI (max number of sem arrays) = 128
shmall [4-KiB pages] = "268435456" (1024.0 GiB max total shared memory)
shmmax [bytes] = "4294967295" (4.00 GiB max segment size)
shmmni [segments] = "4096" (max number of segs)
sysrq [bitmask] = "16" (see proc man page)
fs.
file-max [fds] = "1606351" (system-wide limit for num open files [file descriptors])
nr_open [fds] = "1048576" (per-process limit for num open files [see also RLIMIT_NOFILE])
file-nr [fds] = "5952 0 1606351" (num allocated fds, N/A, num free fds)
inode-nr [inodes] = "84728 46995" (nr_inodes allocated, nr_free_inodes)
net.
core.netdev_budget [packets] = "300"
core.netdev_max_backlog [packets] = "1000"
core.rmem_default [bytes] = "212992" (208 KiB)
core.wmem_default [bytes] = "212992" (208 KiB)
core.rmem_max [bytes] = "212992" (208 KiB)
core.wmem_max [bytes] = "212992" (208 KiB)
ipv4.icmp_echo_ignore_all [bool] = "0"
ipv4.ip_forward [bool] = "1"
ipv4.ip_local_port_range [ports] = "32768 61000" (defines ephemeral port range used by TCP/UDP)
ipv4.ip_local_reserved_ports [ports] = "" (comma-separated ports/ranges to exclude from automatic port assignments)
ipv4.tcp_max_orphans [sockets] = "65536" (4096 MiB @ max 64 KiB per orphan)
ipv4.tcp_mem [4-KiB pages] = "376863 502487 753726" (1.44 GiB, 1.92 GiB, 2.88 GiB)
ipv4.udp_mem [4-KiB pages] = "378810 505082 757620" (1.45 GiB, 1.93 GiB, 2.89 GiB)
ipv4.tcp_window_scaling [bool] = "1"
ipv4.tcp_rmem [bytes] = "4096 87380 6291456" (4 KiB, 85 KiB, 6144 KiB)
ipv4.tcp_wmem [bytes] = "4096 16384 4194304" (4 KiB, 16 KiB, 4096 KiB)
ipv4.udp_rmem_min [bytes] = "4096" (4 KiB)
ipv4.udp_wmem_min [bytes] = "4096" (4 KiB)
ipv4.tcp_sack [bool] = "1"
ipv4.tcp_timestamps [bool] = "1"
vm.
dirty_ratio = "40" (% of total system memory)
dirty_bytes = "0" (disabled -- check dirty_ratio)
dirty_background_ratio = "10" (% of total system memory)
dirty_background_bytes = "0" (disabled -- check dirty_background_ratio)
dirty_expire_centisecs = "3000"
dirty_writeback_centisecs = "500"
nr_hugepages [2-MiB pages] = "0"
overcommit_memory [0-2] = "0" (heuristic overcommit)
overcommit_ratio = "50"
oom_kill_allocating_task [bool] = "0" (scan tasklist)
panic_on_oom [0-2] = "0" (no panic)
swappiness [0-100] = "10"
Throughout the book, we look at using /proc/sys kernel tunables ... this is another thing xsos can help demystify a bit. It has a --sysctl (-s) option that prints out some of the most interesting (to me) sysctls with a little explanation.
Virtually every sysctl mentioned in the book (except the kernel.sched.* ones) is present, with proper unit conversion/explanation where necessary.
Other goodies
There's also simple binary-graphing of /proc/interrupts and quite a lot of other things.
Conclusion
So obviously this tool isn't in RHEL and won't be on the exam; however, it's used by Red Hat's global support organization extensively, as well as by some of our biggest (TAM) customers. Whether and how the curriculum team chooses to use it ... I leave up to you.
