When installing R 3.4.1 on Windows 7 where R 3.3.3 was installed, the result after that installation was that the control panel showed both versions as being installed. After removing the older version via control panel, the file associations for the newer version got lost. So I removed the new installation also (to reinstall it).
Most other software either insist on removal of a previous version, or do that automatically, or maybe even do some smart upgrade of the software. I think R should do likewise.
Note: Bugzilla doesn't yet know about version 3.4.1 ;-)