Investigation of microscopic instabilities in fiber-reinforced composite materials by using multiscale modeling strategies