Yep, and that makes implementing addition of "Connection: close" in an HTTP reply at the HTTP/1.1-server's side somewhat tricky: you ideally need to read all of the pipelined requests from the client before closing the connection, which is usually something you'd rather not do. But if you just close it, you risk your client getting a partial reply, so you better add "Content-Length"/"Transfer-Encoding: chunked" in your reply as well... but one common reason to do connection-close reply is when you don't know the content-length beforehand, so — I hope you implemented chunking correctly :)
Even if you just close it, and it RSTs, and you implement chunked etc. properly, the client can't even necessarily read all of the information already sent to the client.
If the client does a read() after the RST gets there, any data in the receive window is gone, any packets that might have been dropped and need to be resent are gone, etc.
More explicit connection closing indications are one of the nice things of http/2. Of course, it's bundled with the silly multiplexing :(