Got to love the pseudo markup slop! An id attribute on an XML closing tag?!? Complete nonsense. Working nonsens, of course, but still nonsense.
Got to love the pseudo markup slop! An id attribute on an XML closing tag?!? Complete nonsense. Working nonsens, of course, but still nonsense.
> Working nonsens, of course
Well, maybe? There is a lot of valid XML ingested in the training data, so I wonder what happens when the model encounters:
Surely whatever is putting in the <pasted_content ...> tags is also escaping the pasted content with e.g. < to <
Put in a CDATA section and all bets are off. Maybe that is the next benchmark? Parse this XML correctly. Oh, by the way it must be valid, and here is a DTD. Using code is cheating.
But <![CDATA[xyz]]> would become <![CDATA[some stuff]]>
It's basically just a MIME boundary but in a pseudo-XML format which the model understands more readily. Seems pretty reasonable to me, even though it may not be an ideal solution in every regard.
I've switching from only using markdown in my prompts to using XML tags this year too. It's not only easy for the model to see when something ends, it's quite useful for me too.
in retrospect though, how many malformed 3-column website layouts could we have avoided with this technology? :-)