We didn't threaten the OW with harassment charges, but we did surprise her with a restraining order. I'll keep my story short though.
At first she kept calling. She would always block her number, called every hour, and at odd times. We would get 5 to 10 calls a day. Since we really couldn't prove that it was her, we called the phone company, had them put a trace on our phone line, which kept track of the numbers that were calling our house.
In order to do this, we needed to call the police and file a "harassing phone call" complaint. The phone company wouldn't do this without a case number from the police.
The trace was put into effect for 21 days, and when it expired, I was sent a report informing me of any information that was found. Unfortunately, in my case, I believe that the OW has a police scanner, heard the dispatch, and as soon as I had called the phone company, the harassing calls stopped. But, being that she was also using emails to harass us, we used those instead for evidence (but that's another story).
So, what I'm thinking here is that you could go the same route, get the report back from the phone company and then seek out legal counsel. I don't know if it would be enough to get a restraining order. Perhaps you can call your local police department and ask their advice and see what steps you need to take? I wouldn't threaten the other person with harassment charges directly, since that would involve contact.
Your situation probably isn't as bad as ours was, and you probably don't have to take it as far.
In fact, the OW is still "indirectly" threatening us, even though the restraining order is in effect for 2 years. Apparently, she has found a way around it.
Good luck and I hope I've helped somewhat.