Man, I feel that frustration deep in my bones. I used to get so mad when apps would call my garnet mica schist just plain shale. What finally clicked for me was using Geology Toolbox like bennett.vera said, but really leaning on its texture gallery. I'd take five pictures from different angles and compare them all to their example photos, not just the first match. It's still not perfect, but that method got me to correctly ID gneiss about seven times out of ten, which felt like a huge win.
Taking pictures from different angles is definitely the way to go. The lighting can really change how the schist's layers look, which helps the app catch details. It's a bit more work, but it beats getting shale as an answer every time.