Why a mining rig freezes solid — and what to do about it

First, figure out which picture you have

"The rig froze" describes three different faults with different culprits. Check which one is yours:

Power delivery — suspect #1

If the whole rig goes down regularly, start with power. A multi-GPU rig is several hundred watts per card plus a start-up peak when all cards wake at once. Typical mistakes rigs pay for overnight:

• One cable chain feeding several cards. Cables and PSU connectors are rated for a specific current: what "held two cards" last year may fail at tomorrow's peak. Each heavy card deserves its own cable run from the PSU — not a three-way splitter.
• Molex→SATA adapters on risers. SATA connectors are not built for riser current; melted and scorched adapters are a standard find. Power risers with a proper PCIe cable off the PSU.
• Extension cords and cheap power strips. Their contact groups are consumables. Try a direct connection or a decent strip for a week and compare.
• No headroom. A PSU sized exactly to the cards' limits runs on the edge, especially in summer heat and on days with mains voltage dips. Measure the actual peak with a wall wattmeter and keep a reserve.

Cheap and fast: plug a household wattmeter into the wall for 20–30 minutes at peak. One such check often replaces a week of guessing.

Risers and slots — suspect #2

Cheap USB risers and crowded motherboards are the second most common source of overnight silences. Checks, in order:

• Power each riser from its own PCIe cable off the PSU (see the adapters note above).
• Reseat the riser and the card, then try a neighbouring slot. If the problem "follows" the riser — the riser is guilty; if it stays on the slot — look at that slot's power line or the motherboard.
• Budget risers from the same thin batch tend to fail in batches: if a couple of them came together, replace both rather than only "the one that looks worse".
• Do not forget seating and contact quality: a tilted riser or an oxidized connector gives exactly the same result — a rig that is silent at night and looks normal by day.

Hung processes and the annoying truth about HiveOS logs

The most deceptive case: the miner process ends up in a state where the rig looks alive over SSH, but the miner itself does not answer and cannot be killed the usual way — such a process holds the kernel in uninterruptible wait, and nvidia-smi stops responding along with it. There is exactly one treatment: a forced reboot.

Right next to it sits the most frustrating HiveOS trap: system logs live in RAM. A normal reboot wipes them along with the cause. Unless you enabled persistent logging (logs-on or a dedicated log partition), you are not investigating "yesterday's" hang — you are investigating a mystery with no evidence. Turn on log persistence before the next failure; after the reboot there is nothing to turn it on for.

And miner versions matter: updates sometimes change driver behaviour on the very same cards. If hangs started with a fresh miner release, read other people's reports carefully before blaming your hardware.

Checklist: look for a pattern, not a miracle

1. Stock presets for a week

   Put the cards back to stock (or safe factory profiles). Hangs on stock? Then overclocking is not the culprit at all — it is hardware, power, or software. No hangs on stock but hangs on your profile? Your search area is found — trim offsets back, one card at a time.

2. One card at a time

   The practical rule: run on the minimum set of cards for a few days, adding one card per day. When the hangs return, the culprit sits in the last added combination (card + riser + slot).

3. Wattmeter at peak

   Compare the actual maximum against the PSU rating and the state of the line.

4. Rotate risers

   Swap risers across slots and cards: whatever travels with the card is the card; whatever stays with the slot is the riser or its power.

5. Logs — in advance

   Enable persistent logging before the next failure; the trail is lost after a reboot.

6. Do not leave the rig silent until morning

   Once you know the likely causes, act on them: the rig needs something that notices a failure before you do and does not sleep at night. That job can be done by automation.

What is worth handing to automation

Even a farm with spotless hardware leaves two "human" problems: failures happen at night, and a manual decision about a reboot means a drive to the farm or a wake-on-LAN round trip — always slower than losing a few minutes of hashrate. Automation solves exactly this:

• rig status and failures arrive in Telegram instead of being "discovered in the morning";
• one-off glitches are told apart from real problems: no reboots over trivia, and a real failure is handled without driving to the farm;
• the rig is back to work within minutes — a setting tweak, a miner restart, a reboot, or a rollback to the last verified preset, depending on what actually happened.

That is exactly what OCBot does: the bot runs on the rig itself, no external server, with control and reports in Telegram. It also tunes a stable overclock for each card on its own — verified over long runs, not "it worked for five minutes". A 3-day trial is enabled automatically at install, no payment needed.