I spent about an hour looking at the code and found some glaring issues that should be fixed before trusting it with real money.
- Yahoo News is introduced twice (sentiment and news analysis) which double weights it
- Sentiment analysis prompt primes the model to be bullish on Nvidia.
- In the self learning loop there is a complex parsing bug that results in hallucinated memories when agents return truncated responses
- You can completely control sentiment analysis of a subreddit by simply maintaining a majority of the 5 most recently posted messages, regardless of any quality metric
- The reflection prompt states the agent must cite alpha, which in a market wide downturn causes it to think correctly placed calls were losses
There is also problem with underlying models. There was a study, where they always repest the same investing/management strategy: trust strangers, be open minded/adopt to new unproven ideas, prefer cooperation... Basically they were trained on disney-boomer bull(shit) market of last 15 years.
They have zero guards against market manioulations, and will get wiped without bull market!
> Sentiment analysis prompt primes the model to be bullish on
Eeh, yeah? At that point I'd stop reading the code and just leave the project behind. How exactly is the prompt doing this right now?
[flagged]