"Is there any enforceability behind a boss saying "I don't accept your resignation"?" - no.

The boss can say this meaning "I would really like you to think about this because I don't want to let you go", but that won't have any legal value - unless you are required to give notice and didn't give enough notice. In some countries (India, for example), the boss can play all kinds of shenanigans to stop you from leaving, which may be more or less illegal.

With the boss not recognizing the resignation, my concern would be leaving on a good note. I'd tell your friend to raise the concern with HR and ensure you put in a good 2 weeks notice so that you're leaving on a good note.

For example suppose that for whatever reason written resignation is required by contract, and you were to hand your boss a blank sheet of paper while saying, "I resign". Then the boss quite reasonably could say "I don't accept your resignation [since it isn't written]", and expect that in any court case, provided the contract was upheld, it would be deemed that no resignation had been submitted. Whether a clause in an employment contract that requires written resignation, is actually enforceable, would be a legal question :-)

@gnasher729 - In some countries (India, for example), the boss can play all kinds of shenanigans to stop you from leaving - could you please elaborate?

A boss can enforce their refusal to accept a resignation by making a counter offer that is too good to refuse.

Your friend should've asked why... This is such a vague statement.

@rohithpr Some examples on What is a Relieving Letter? and the questions in that tag. There's probably more things.

@SteveJessop I thought that the idea of handing a blank sheet of paper was cute, it is like saying: "here is my new contract." The problem is, the employer could then write anything they wanted on it and it would be binding! (Only if you signed it, I guess.) Think of it as a blank un-check.

This question is incomplete - why did the boss reject his resignation? It's not common to reject a resignation, but if we know why the boss thinks he can then we can say whether that reason is valid. One that I can think of is the employee was offered training that required him to stay for a certain amount of time. That might require him to reimburse the company for that training if he were to leave before that time.

@Emory is correct; usually it is "I don't accept your resignation, we need you, and we'll fix whatever is compelling you to leave". That allows the employee and the employer to begin a round of negotiations that maybe makes everyone happy. The President saying "I don't accept your resignation soldier!" and expecting the employee to keep on being Secretary of Defense only happens in movies.

Texas uses prisoners as forced labor (e.g. Texas Corrections Industries faces prisoner strike ). If this was a puzzle question "the boss didn't accept his resignation, how can this happen?", I'd guess the friend was both prisoner and employee, did have a boss, jokingly handed over a letter of resignation and the boss didn't accept it. Can a boss enforce their refusal to accept a resignation? - in Texas, in certain circumstances, it can happen. (But really, I think you'd have mentioned that kind of situation).

If the answer would be "yes" it wouldn't be called employment, it would be called slavery

@dirkk ...perhaps more indentured servitude

I had this happen to me once. I responded, "Well, two weeks from today, I'm going to stop showing up or doing work. You can have me use those two weeks to smooth this for you or not, it's your choice. And after that, you can keep paying me or stop paying me. That is also your choice."

The short answer here is "No, they cannot, at least not in the U.S.". Think about it for just a second. If an employer could refuse resignations and enforce their refusals, that effectively means they have the power to compel people to work against their will. That pretty much literally amounts to slavery. Which is something the U.S. fought a war over, and then declared unconstitutional.