Names containing "`" are allowed, but the parser and deparser do not handle this case:
> assign("`", TRUE)
Error: attempt to use zero-length variable name
Error: '\`' is an unrecognized escape in character string starting "`\`"
The parser is behaving as documented, but it seems reasonable to add \` as an allowable escape, and use it in printing. I'll put this change into R-devel and R-patched.