Funny, gemma will do this all day long.
>>> Please write a hello world app in java
```java
public class HelloWorld {
public static void main(String[] args) {
System.out.println("Hello, World!");
}
}
```
https://docs.oracle.com/javase/tutorial/getStarted/applicati...cat /bin/true and /bin/false if you are on a Solaris etc... as an example too.
Note this paper that will be presented at ICSE in a couple of weeks too.
https://arxiv.org/abs/2408.02487v3
The point being is that this is very much a very real and yet unsolved problem with LLMs right now.