Fermions and scalars in N=4 Wilson loops at strong coupling and beyond