They can do new tasks with in-context learning but its obviously limited by context window