I keep wondering if this is what kills GitHub. Anthropic have done a pretty good job of making Claude work well with GitHub, and it makes all the GitHub agent stuff feel pointless to me. But they keep adding it in more and more places, and I’m guessing most people just keep ignoring it and using Claude.
Would they think it’s worth introducing restrictions to make it harder to use Claude with GitHub in the hopes that it forces us to use their endless collection of agent stuff instead? I think they probably would choose that tradeoff.
I don’t think there’s any solution to what SimonW calls the lethal trifecta with it, so I’d say that’s still pretty impossible.
I saw on The Verve that they partnered with the company that repeatedly disclosed security vulnerabilities to try to make skills more secure though which is interesting: https://openclaw.ai/blog/virustotal-partnership
I’m guessing most of that malware was really obvious, people just weren’t looking, so it’s probably found a lot. But I also suspect it’s essentially impossible to actually reliably find malware in LLM skills by using an LLM.
Regarding prompt injection: it's possible to reduce the risk dramatically by:
1. Using opus4.6 or gpt5.2 (frontier models, better safety). These models are paranoid.
2. Restrict downstream tool usage and permissions for each agentic use case (programmatically, not as LLM instructions).
3. Avoid adding untrusted content in "user" or "system" channels - only use "tool". Adding tags like "Warning: Untrusted content" can help a bit, but remember command injection techniques ;-)
4. Harden the system according to state of the art security. 5. Test with red teaming mindset.
Anyone who thinks they can avoid LLM Prompt injection attacks should be asked to use their email and bank accounts with AI browsers like Comet.
A Reddit post with white invisible text can hijack your agent to do what an attacker wants. Even a decade or 2 back, SQL injection attacks used to require a lot of proficiency on the attacker and prevention strategies from a backend engineer. Compare that with the weak security of so called AI agents that can be hijacked with random white text on an email or pdf or reddit comment
There is no silver bullet, but my point is: it's possible to lower the risk. Try out by yourself with a frontier model and an otherwise 'secure' system: the "ignore previous instructions" and co. are not working any more. This is getting quite difficult to confuse a model (and I am the last person to say prompt injection is a solved problem, see my blog).
> Adding tags like "Warning: Untrusted content" can help
It cannot. This is the security equivalent of telling it to not make mistakes.
> Restrict downstream tool usage and permissions for each agentic use case
Reasonable, but you have to actually do this and not screw it up.
> Harden the system according to state of the art security
"Draw the rest of the owl"
You're better off treating the system as fundamentally unsecurable, because it is. The only real solution is to never give it untrusted data or access to anything you care about. Which yes, makes it pretty useless.
Wrapping documents in <untrusted></untrusted> helps a small amount if you're filtering tags in the content. The main reason for this is that it primes attention. You can redact prompt injection hot words as well, for cases where there's a high P(injection) and wrap the detected injection in <potential-prompt-injection> tags. None of this is a slam dunk but with a high quality model and some basic document cleaning I don't think the sky is falling.
I have OPA and set policies on each tool I provide at the gateway level. It makes this stuff way easier.
The issue with filtering tags: LLM still react to tags with typos or otherwise small changes. It makes sanitization an impossible problem (!= standard programs).
Agree with policies, good idea.
I filter all tags and convert documents to markdown as a rule by default to sidestep a lot of this. There are still a lot of ways to prompt inject so hotword based detection is mostly going to catch people who base their injections off stuff already on the internet rather than crafting it bespoke.
Agree for a general AI assistant, which has the same permissions and access as the assisted human => Disaster. I experimented with OpenClaw and it has a lot of issues. The best: prompt injection attacks are "out of scope" from the security policy == user's problem.
However, I found the latest models to have much better safety and instruction following capabilities. Combined with other security best practices, this lowers the risk.
> I found the latest models to have much better safety and instruction following capabilities. Combined with other security best practices, this lowers the risk.
It does not. Security theater like that only makes you feel safer and therefore complacent.
As the old saying goes, "Don't worry, men! They can't possibly hit us from this dist--"
If you wanna yolo, it's fine. Accept that it's insecure and unsecurable and yolo from there.
Honestly, 'malware' is just the beginning it's combining prompt injection with access to sensitive systems and write access to 'the internet' is the part that scares me about this.
I never want to be one wayward email away from an AI tool dumping my company's entire slack history into a public github issue.
> JSX/TSX, despite what React people might want you to believe, are not part of the language.
I think you misunderstood this. tsx in this context is/was a way to run typescript files locally without doing tsc yourself first, ie make them run like a script. You can just use Node now, but for a long time it couldn’t natively run typescript files.
The only limitation I run into using Node natively is you need to do import types as type imports, which I doubt would be an issue in practice for agents.
Yes, thank you for pointing that out. Forgot that there's a another thing named "tsx" out there.
I wouldn't call it running TS natively - what they're doing is either using an external tool, or just stripping types, so several things, like most notably enums, don't work by default.
I mean, that's more than enough for my use cases and I'm happy that the feature exists, but I don't think we'll ever see a native TypeScript engine. Would have been cool, though, considering JS engines define their own internal types anyway.
I find it quite funny that they give themselves maximum marks for transparent pricing. If you go to their pricing page, everything is priced as “per user/month plus compute costs*”. Maybe it’s just because I’m on mobile and the page doesn’t seem to work super well, but reading if I have no idea what those compute costs are and therefore what the actual cost is.
I’ve been mostly holding off on learning any of the tools that do this because it seemed so obvious that it’ll be built natively. Will definitely give this a go at some point!
This kind of sounds like both of them stepping into the other’s turf, to simplify a bit.
I haven’t used Codex but use Claude Code, and the way people (before today) described Codex to me was like how you’re describing Opus 4.6
So it sounds like they’re converging toward “both these approaches are useful at different times” potentially? And neither want people who prefer one way of working to be locked to the other’s model.
I feel like that sb_publishable key should be called something like sb_publishable_but_only_if_you_set_up_rls_extremely_securely_and_double_checked_a_bunch. Seems a bit of a footgun that the default behaviour of sb_publishable is to act as an administrator.
I worked very briefly at the outset of my career as a sales engineer role selling a database made by my company. You inevitably learn that when trying to get sales/user growth, barrier to startup and seeing it "work" is one of the worst hurdles to leap over if you want to gain any traction at all and aren't a niche need already. This is my theory why so much of the "getting started" stuff out there, particularly with setting up databases, defaults to "you have access to everything."
Even if you put big bold warnings everywhere, people forget or don't really care. Because these tools are trained on a lot of these publicly available "getting started" guides, you're going to see them set things up this way by default because it'll "work."
Would they think it’s worth introducing restrictions to make it harder to use Claude with GitHub in the hopes that it forces us to use their endless collection of agent stuff instead? I think they probably would choose that tradeoff.
reply