centos field guide
EOL era · run Stream

Field manual

Trenches & Truths

Things we learned the expensive way, so you don’t have to. Scroll — the cards stack, and the checklist remembers what you tick.

A dim server corridor
WHERE THE TRUTHS CAME FROM
01

Seven things we know now

Cards stack as you scroll — each one a scar that became a rule.

01

Update first, always

Run sudo dnf update -y before installing anything new. On Stream you are tracking a moving target — a fresh base means dependency resolution starts from a clean, known state instead of a three-month-old drift.

sudo dnf update -y
02

There is an undo button

dnf keeps a numbered transaction log. An upgrade that broke something? You do not reinstall — you roll that one transaction back. Check the history before the panic sets in.

dnf history list · sudo dnf history undo <N>
03

Don’t switch SELinux off

setenforce 0 hides the disease. Read the AVC denials instead, then bend the policy with booleans or a context fix. Permissive mode is a diagnostic, not a destination — restore Enforcing when you leave.

ausearch -m avc -ts recent
04

htop is not top

Install htop and never look back: F6 sorts by any column, Shift+P sticks the top process on top, F9 sends signals. Compare the LOAD column against your CPU count — that ratio tells the whole story.

htop  (F6 sort · Shift+P pin · F9 kill)
05

journalctl is a time machine

“What happened last Tuesday at 3 a.m.?” — you can ask. Filter by time, by unit, by priority, and follow live. The old habit of greping /var/log/messages is a fossil; the journal is structured and queryable.

journalctl --since "2 hours ago" -p err
06

Cron is still king

Nothing beats a small cron job for “tell me before it becomes an incident”. Fifteen minutes a day, du on the big directories, and an email when numbers cross a line. Boring, reliable, undefeated.

15 3 * * * du -xsh /var/* >> /var/log/diskwatch 2>&1
07

Know your repos

Stream is rolling — a “stable” upgrade can still land a new kernel under you. Know exactly what dnf repolist shows, and pin the few packages where a surprise version matters: an explicit dnf install kernel-6.12.0-114* beats a vague “latest”.

dnf repolist · exclude=kernel in /etc/dnf/dnf.conf
02

SELinux without tears

Symptom → diagnosis → the command that actually fixes it.

SymptomDiagnoseFix
nginx serves 403 on static filesls -Z /var/www/html — context is not httpd_sys_content_tsudo restorecon -Rv /var/www/html
Samba can’t read home dirsgetsebool samba_enable_home_dirs → offsudo setsebool -P samba_enable_home_dirs 1
App can’t open outbound connectionsgrep avc /var/log/audit/audit.log | tailsudo setsebool -P httpd_can_network_connect 1 or build a policy module
Port refuses to listenss -ltnp + audit denials mentioning the portfirewall-cmd --add-port and the matching sebool
03

“It’s not reachable” — a flow

Five questions, in order. The answer is almost always #2 or #4.

1

ping

Does the host answer at all? Silence may be an ICMP block — don’t conclude from silence alone.

2

ss -tlnp

Is anything listening — and on 0.0.0.0, or only 127.0.0.1? A localhost-only bind is the classic trap.

3

firewall-cmd

--list-ports and --list-services: is the port actually open to your source subnet?

4

audit.log

grep avc — is SELinux eating the connection? Denials show up here with the full story.

5

journalctl -u

Follow the daemon live while you retry: journalctl -u httpd -f. Let it tell you what it thinks.

04

Hardening checklist

Tick as you go — your progress is saved in this browser and survives reloads.

Server hardening · 10 controls

05

Reading logs like a pro

Which file, and when it actually matters.

SourceWhat it holdsWhen to look
journalctlEverything, structured, queryable by time/unit/priorityAlways — it is the front door
/var/log/dnf.logEvery install/upgrade transaction, with timestampsAfter updates, before blame
/var/log/audit/audit.logSELinux AVC denials, login events, policy changesAny “permission denied” with no explanation
/var/log/cronCron job invocations and their stderr“The job didn’t run” — it usually ran and died
/var/log/last · btmpSuccessful and failed login attemptsAfter any suspected intrusion
~/.dnf/historyPer-user dnf transaction historyUndo decisions, “who upgraded what”

# /var/log/messages is legacy — on Stream, journald owns the story.