A search for decays of the Higgs boson into a pair of new spin-zero particles, H→aa, where the a-bosons decay into a b-quark pair and a muon pair, is presented. The search uses 36.1 fb−1 of proton–proton collision data at s=13 TeV recorded by the ATLAS experiment at the LHC in 2015 and 2016. No significant deviation from the Standard Model prediction is observed. Upper limits at 95% confidence level are placed on the branching ratio (σH/σSM)×B(H→aa→bbμμ), ranging from 1.2×10−4 to 8.4×10−4 in the a-boson mass range of 20–60 GeV. Model-independent limits are set on the visible production cross-section times the branching ratio to the bbμμ final state for new physics, σvis(X)×B(X→bbμμ), ranging from 0.1 fb to 0.73 fb for mμμ between 18 and 62 GeV.