Some people end up putting the screwdriver through the oil cooler as well.
Oh man, that would be brutal. I'd cry.
They're on there really tight from the factory, a lot of people torque them to the specs listed in the manual. I don't torque mine super tight but you definitely wouldn't get it off without a filter wrench, or the screwdriver method you used.
I have a filter wrench, even that wasn't enough to get it off. Whoever put it on tightened the hell out of it, way past manual specs. There's no point in over-tightening anyway, it'll bottom out against the metal and you wont crush the o-ring any tighter.