I would argue that any given technology tends to improve on an S curve, so exponentially at first and then flattening out. See Moore’s law as a great example.
Or more on topic see the improvements in LLMs since they were invented. At first each release was an order of magnitude better than the last (see GPT 2 vs 3 vs 4), now they’re getting better but at a much slower rate.
Certainly feels like being at the top of an S curve to me, at least until an entirely new architecture is invented to supersede transformers.
Or more on topic see the improvements in LLMs since they were invented. At first each release was an order of magnitude better than the last (see GPT 2 vs 3 vs 4), now they’re getting better but at a much slower rate.
Certainly feels like being at the top of an S curve to me, at least until an entirely new architecture is invented to supersede transformers.