The current approach for accurate determination for rare earth elements using environmental reference materials for k0-INAA with short turnaround analysis times was investigated. The k0 and Q0 values for REEs, nuclear interference, and spectral interference were practically assessed and improved with the standard reference material. The evaluation of the current iterative neutron self-shielding and gamma-ray self-attenuation was established using reference material prepared from 100 mg/kg of 99.9% certified reference material (i.e., Alfa Aesar mono rare earth oxide combined with SiO2). The general achievement of the current k0-INAA technique for rare earth elements was validated with IAEA Soil-7 reference material and real sediment samples. The rare earth elements levels in the reference material range from 0.72 ± 0.08 mg/kg for 166Ho to 60.8 ± 2.1 mg/kg for 141Ce. The uncertainty at which the rare earth element concentration was examined is below 8% (at 95% confidence level). This showed good uniformity with the IAEA Soil-7 certified contents. In addition, k0-NAA is able to quantify most of the REEs in real samples collected from three locations in Ghana.