It's currently not possible to re-add a removed file to the list via Undo ⌘ Z — however, you could add it via drag&drop from Finder by holding the ⌥ key down.
Interesting idea, I'm adding this to my internal list!
Would this apply to both files removed from the list, as well as deleted? Undoing a remove would have to retrieve it from the current directory, Undoing a delete would pull it back from the recycle bin.