AFAIK it is already possible to install MP3tag on portable devices as a protable installation and it runs perfectly. And all this works without any need for repackaging. So it "would" not be great, it already is.
I install the non-portable version to my laptop.
You can unzip the setup program (make sure you enable "keep folder structure" option). Copy/paste folders as per Florian suggestion.
I hope this does not break any "law"...
I wish there was a Portable ZIP version or a PortableApps installer version or a Checkbox for making official installer "Portable" (so it won't add anything to Registry or Uninstaller.exe or related files).
The solutions so far presented do not work. The current program will always create the folder %APPDATA%\Mp3tag. This is contrary to what a portable application would do. The whole idea is that you can run your portable programs from a USB stick on a PC that belongs to a friend, and when you are finished there will be no trace of your software on your friend's PC.
We really need an official, and well behaved, version of Mp3tag.
As far as I remember, it would be (technically) possible to create a PortableApps.com-like mp3tag build.
However, I cannot tell whether the PortableApps build brings any infrigement with mp3tag itself (probably not, but better to have an explicit statement).