Wouldn't "Serverless OCR" mean something like running tesseract locally on your computer, rather than creating an AI framework and running it on a server?
Wouldn't "Serverless OCR" mean something like running tesseract locally on your computer, rather than creating an AI framework and running it on a server?
Serverless means spinning compute resources up on demand in the cloud vs. running a server permanently.
~99.995% of the computing resources used on this are from somebody else's servers, running the LLM model.
> Serverless means spinning compute resources up on demand in the cloud vs. running a server permanently.
Not quite. Serverless means you can run a server permanently, but you need pay someone else to manage the infrastructure for you.
You might be conflating "cloud" with serverless. Serverless is where developers can focus on code, with little care of the infrastructure it runs on, and is pay-as-you-go.
> You might be conflating "cloud" with serverless. Serverless is where developers can focus on code, with little care of the infrastructure it runs on, and is pay-as-you-go.
That's not what serverless means at all. Most function-as-a-service offerings require developers to bother about infrastructure aspects, such as runtimes and even underlying OS.
They just don't bother about managing it. They deploy their code on their choice of infrastructure, and go on with their lives.
A runtime is notably NOT infrastructure, had you said instruction set you might have landed closer to making a compelling argument, but the whole point is that AWS (and other providers) abstract away the underlying infrastructure and allow the developers to as I said, have "little care of the infrastructure it runs on". There is often advanced networking that CAN be configured, as well as other infrastructure components developers can choose to configure.
Close. It means there's no persistent infra charges and you're charged on use. You dont run anything permanently.
It still doesn't capture the concept because, say, both AWS Lambda and EC2 can be run just for 5 minutes and only one of them is called serverless.
Unless the engineer takes steps to spin down EC2 infrastructure after execution, it is absolutely persistent compute that you're billed for whether you are doing actual processing or not. Whereas lambda and other services are billed only for execution time.
Depends if you mean "server" as in piece of metal (or vm), or as in "a daemon"
Thanks for noting this - for a moment I was excited.
You can still be excited! Recently, GLM-OCR was released, which is a relatively small OCR model (2.5 GB unquantized) that can run on CPU with good quality. I've been using it to digitize various hand-written notes and all my shopping receipts this week.
https://github.com/zai-org/GLM-OCR
(Shameless plug: I also maintain a simplified version of GLM-OCR without dependency on the transformers library, which makes it much easier to install: https://github.com/99991/Simple-GLM-OCR/)
When people mentions the number of lines of code, I've started to become suspicious. More often than not it's X number of lines, calling a massive library loading a large model, either locally or remote. We're just waiting for spinning up your entire company infrastructure in two lines of code, and then just being presented a Terraform shell script wrapper.
I do agree with the use of serverless though. I feel like we agree long ago that serverless just means that you're not spinning up a physical or virtual server, but simply ask some cloud infrastructure to run your code, without having to care about how it's run.
> When people mentions the number of lines of code, I've started to become suspicious.
Low LoC count is a telltale sign that the project adds little to no value. It's a claim that the project integrates third party services and/or modules, and does a little plumbing to tie things together.
>implement RSA with this one simple line of python!
No, that would be "Running OCR locally..."
'Serverless' has become a term of art: https://en.wikipedia.org/wiki/Serverless_computing
It's good they note explicitly:
> Serverless is a misnomer
Running it locally would typically be called “client(-)side”.
But this caught me for a bit as well. :-)
That's the beauty of such stupid terms.
I use carless transportation (taxis).
taxis are cars, aren't they?
Precisely. And serverless uses servers.
Yep. That fraudulent term finally got me this time. Totally serverless except for that remote 3rd party server. Sigh.