"And hence invertible" <- does every output embedding combination have an associated input ? Are they able to construct it or is this just an existence result ?
"And hence invertible" <- does every output embedding combination have an associated input ? Are they able to construct it or is this just an existence result ?
> Building upon this property, we further provide a practical algorithm, SɪᴘIᴛ, that reconstructs the exact input from hidden activations
I don't think they're claiming surjectivity here. They're just saying the mapping is injective, so for a given output there should be a unique input to construct it.
> I don't think they're claiming surjectivity here.
What definition of invertible doesn't include surjectivity?
Many? Just add a "by restricting to the image, we may wlog assume surjectivity".
The question is usually more about whether the inverse is also continuous, smooth, easy to compute....etc.