We can finally search for playlists with a giving song! A basic feature that Spotify is missing!

So they used a LLM with knowledge cut in mid 2023 to evaluate 2023? Seems like a classic leakage problem.

From paper: "testing set: January 1, 2023, to December 31, 2023"

From the Llama 2 doc: "(...) some tuning data is more recent, up to July 2023."

Removing the "Market expert" which uses OHLCV (Open, High, Low, Close, Volume) also drops the sharpee from 5.01 to 1.88 while also increasing the max draw down to 13.29% (v.s. 9.70% for the index). I'd be very surprised if the pre training of the base model was the only source of leakage...

I think that there may be another solution for this, that is the LLM write a valid code that calls the MCP's as functions. See it like a Python script, where each MCP is mapped to a function. A simple example:

  def process(param1, param2):
     my_data = mcp_get_data(param1)
     sorted_data = mcp_sort(my_data, by=param2)
     return sorted_data

Yes! If you want to see how this can work in practice, check out https://lutra.ai ; we've been using a similar pattern there. The challenge is making the code runtime work well for it.

Not available in my country :(

This is hacker news. Fix it.

On the internet nobody has to know you're in your country.

Obviously ‘they’ do. RIP ‘Inter’net

Unfortunately, it uses Miniconda, which does not allow usage in companies with more than 200 employees. I think it conflicts with AGPL license. I created a PR to fix that.

Thanks for finding this!

Will get this merged as soon as we can test across platforms.

Can you provide more information about this “bigger teacher” model?

Until GPT-4.5, GPT-4 32K was certainly the most heavy model available at OpenAI. I can imagine the dilemma between to keep it running or stop it to free GPU for training new models. This time, OpenAI was clear whether to continue serving it in the API long-term.

It's interesting to compare the cost of that original gpt-4 32k(0314) vs gpt-4.5:

$60/M input tokens vs $75/M input tokens

$120/M output tokens vs $150/M output tokens

> or stop it to free GPU for training new models.

Don't they use different hardware for inference and training? AIUI the former is usually done on cheaper GDDR cards and the latter is done on expensive HBM cards.

Indeed, that theory is nonsense.

Any chance you could release the dataset to the public? I imagine NewsCatcher and Polymarket might not agree..

Co-founder of NewsCatcher (YC S22). There are some reasons for not having a dataset fully open sourced.

But we have free/very very low tiers for academia.

So in case you need access for your research, go to https://www.newscatcherapi.com/free-news-api

Or feel free to email me directly at artem@newscatcherapi.com

It remembers me Theano [0].

[0] https://en.wikipedia.org/wiki/Theano_(software)