Decoherence in a Fermion Environment: Non-Markovianity and Orthogonality Catastrophe