CGI and FastCGI are two different things in two different domains. Well the domains are not that different but enough that CGI solves a real problem and makes sense and FastCGI does not. CGI is the interface between a HTTP transaction and a process. It answers the question "How do we turn a HTTP request into executing a process?". FastCGI answers the question of "How do we turn a HTTP request into a FastCGI request". a convolution that leaves you asking "Why are we jumping through this hoop? Is FastCGI actually bringing anything to the table?, Is it actually more difficult to have a HTTP server instead of a FastCGI server if they are so trivially connected?
I am halfway convinced the only reason FastCGI exists is we had got in a mindset that executable code in a HTTP context had to run via the Common Gateway Interface and when we wanted to to change to a persistent process model it had to have the CGI name as well. Well FastCGI to the rescue it does exactly what HTTP does but is not HTTP and most importantly has CGI in the name.
As to the articles complaint, "A HTTP relay server had a bug. Therefore HTTP is intrinsically bad". Well.. it failed to convince me. I am not exactly in that domain(backend web development) so my view is not worth much. But I feel that your internal HTTP(application) servers should be built as if they were going directly on the open web. Then you put some relay servers in front in order to block, balance and route requests. But avoid putting too many smarts in the relay servers. A smart network is almost always a bad idea. try and stick with a dumb network and smart edges.
> Is FastCGI actually bringing anything to the table?,
Yes; it removes the need for the application server to perform parsing of HTTP, which is notoriously difficult to consistently do safely. The application server can use the safer and simpler FastCGI protocol rather than try to support the full HTTP spec.
> Is it actually more difficult to have a HTTP server instead of a FastCGI server if they are so trivially connected?
IME, yes. HTTP (the spec) is full of footguns. FastCGI has fewer footguns. HTTP requires a long dependency chain in your application. FastCGI requires maybe a single library.
> But I feel that your internal HTTP(application) servers should be built as if they were going directly on the open web.
I feel that too, but which developer do you know writes their own HTTP server inside their application? They all use the most popular server via a library or framework, almost all of which warn not to open that to the public internet.
What do you propose they do? They use a framework/library and get a warning not to expose it to the public internet, they don't use a framework/library and odds are good that they coded some vulnerability into it.
Your question is why we have http servers and http middleware at all. Running an application directly on an endpoint makes sense for embedded application, but for everything else you will end up with a separate http terminating layer for a number of reasons.
It makes maintenance so much easier, when you can scale request backends independently of request frontends. It makes sense that the application doesn't have access to TLS keys and can't bind end user facing ports directly. The day you want a separate access log from the application log, and separate instrumentation of the frontends because of some hard to track down bug you will be thankful it is there.
So, you want to deploy some sort of proxy. Will you run FastCGI, SGI, WSGI, uWSGI, plain http, or a number of proprietary load balancer protocols? If you choose http you have to take care to wash your headers and be very careful with pipelining so front and back end expect the same behaviour (which is true for several of the proprietary protocols too). Otherwise there can be catastrophic security implications.
It is probably true that FastCGI was designed as a persistent CGI. But as soon as you have persistence, the http server needs to forward transactions to the persistent backend, and what you have built is a proxy. There is no way around it.