No. Most transformer variants work on either a flat stream, or a fixed sized internal state.