Illegal instruction crash from a native vector DB module on an old Xeon

Problem

AnythingLLM's backend crashed with Illegal instruction (core dumped) every time a document was embedded. The container restarted cleanly and served API calls fine — the crash only happened on the first vector write.

Investigation

The crash was reproducible on the ingest path but never at boot. The default vector database (LanceDB) ships a native Rust module that loads lazily on first write. The host is a Xeon E5-2697 v2 (Ivy Bridge, 2013) — it lacks AVX2, which modern Rust binaries are often compiled against. The first write executed an unsupported instruction and the process died with SIGILL.

Key evidence:

• Boot and API calls worked (the native module isn't loaded until first use).
• docker logs showed the crash only after the embedding call.
• The same flow worked on a newer machine.

Fix

Switched the vector database from LanceDB to Qdrant, which is a separate HTTP service — no native module in the app container:

environment:
  - VECTOR_DB=qdrant
  - QDRANT_ENDPOINT=http://100.86.150.70:6333

Qdrant was already running in the lab stack, so this was a config change, not a new dependency.

Verification

• Ingest succeeded: documents appeared in the workspace and in the Qdrant collection.
• Retrieval returned correct answers with sources.
• No further crashes.

Lesson

When a containerized app crashes with Illegal instruction on older hardware, suspect native modules (Rust/C++ bindings) compiled for newer instruction sets. The fix is usually to swap the native dependency for a service over HTTP — which also keeps the app container smaller and the data plane separate.