It's not testing the API for bugs, it's asking whether the API you have follows typical norms and isn't missing things. Is your API confusing or incomplete?
Imagine you have a library dealing with chat rooms, and LLMs keep trying to call "room.getUser(id)" and "room.getUsers()" when your API is actually "room.user(filter:{id:id})[0]" and "room.user(filter:None)". Maybe that's a sign that your API, while totally functional is going to be confusing to other programmers. Maybe you don't have anything that lists all current users, or there's a flag or option somewhere that would make sense or be more typical.
A fuzzer won't tell you that.
For the original post, you could imagine that a user might want the price of a token in a different currency, but maybe it currently returns everything in dollars. An LLM might try and add ?currency=GBP . Maybe an LLM keeps expecting to see in the response data the last updated timestamp to check data freshness, but it isn't something you add right now.
fair point again. i dont think this is the way to go in either case. of your api is easy to clearly document its good. if you struggle with that, a programmer trying to call it will also. regardless of what an LLM makes of it.
it seems like trying to facilitate bad things. make good things instead.
> of your api is easy to clearly document its good
If it breaks a lot of conventions or norms, no it's probably not good.
But also the point here is *how much do you need to really explain?". If you have apis that require a lot of re-explanation to llms to force them to do the right thing "no getUsers doesn't exist use users(filter) where... YOU MUST USE users(filter)!" That's a sign your API might be going counter to a broad estimation of existing code.
> it seems like trying to facilitate bad things. make good things instead.
I have seen so many people make something that is obvious to them and so few others that "simply write good things from the start" is pointless advice.
make good things, shouldn't be based on a sole person's opinion i suppose. I didn't want to elaborate on what that is, but as you pointed out, use standards and conventions, well documented and tested libraries. Don't stray from the common thing if it's not needed. Reason about data first, then code. - these are all common advices too, easy to find if you look on any search engine how to build something 'good' or 'properly'. or 'how to program professionally'. - i'd hope someone uses the resources and knowledge available to them that others have spent a lot of time documenting and building.
That being said, its not always the case and you are right, 'make something good' is a bad advice, essentially it's not advice at all.
I would say as advice - don't do something alone if you want others to use it. Try to get feedback as early as possible from people you trust, and from people you'd expect to be your consumers/users. And always listen to that advice, don't throw it into the wind. I think that's the best advice if you want to make something 'successful', because the label of successful ultimately will be granted to your product by your users, not yourself.
Sorry to put a 'bad advice' if it was taken as advice :-). I don't mean to make bad things even in advice ^^.
It's not testing the API for bugs, it's asking whether the API you have follows typical norms and isn't missing things. Is your API confusing or incomplete?
Imagine you have a library dealing with chat rooms, and LLMs keep trying to call "room.getUser(id)" and "room.getUsers()" when your API is actually "room.user(filter:{id:id})[0]" and "room.user(filter:None)". Maybe that's a sign that your API, while totally functional is going to be confusing to other programmers. Maybe you don't have anything that lists all current users, or there's a flag or option somewhere that would make sense or be more typical.
A fuzzer won't tell you that.
For the original post, you could imagine that a user might want the price of a token in a different currency, but maybe it currently returns everything in dollars. An LLM might try and add ?currency=GBP . Maybe an LLM keeps expecting to see in the response data the last updated timestamp to check data freshness, but it isn't something you add right now.