I don't think you're doing anything wrong, but at this point it could just be luck. Every time I've caught the oarfish was when I thought, by its shadow, that it was a tuna.
I must've been having good luck the day I got mine. I scared off a fish I was sure was an oarfish so I was annoyed. Then I went to the other side of the beach and saw a good size shadow. It was a tune. Right after I caught it another big shadow appeared and it turned out to be a coelacanth! I'll take those fish any day. Not long after, back on the other side of the beach I got my oarfish(only fish I need now is a loach). That was a good day fishing. You'll catch one just keep at it. It took me ages to get a bagworm, that was sad.
It's all due to luck when you catch them as the big shadows you see in the sea could be anything .... I've caught the oar fish but the shadow was the same size as every other big fish .....just like the small shadows from the river I caught a goldfish and yet I caught a carp which was much bigger in the same shadow.