That's a great question - right now it is only looking at the results from a battery of several dozen indicators that we compute upstream of the model itself (which saves massively on tokens)
As small models continue to improve, and edge hardware becomes more capable, we would really like to run larger models that could incorporate full page content and screengrab data, which would be more likely to catch these kinds of attacks.
But we also find that sites that do one shady thing usually do others, which is a big reason why a tiny model like this can work - and why we are betting on low latency being a differentiating factor in real-world impacts.
CLAVIER is amazing, the wire system alone is such a huge improvement over ORCA, and it's now feasible to make much larger patches and refactor safely, kudos to River for all the hard work on the polish and quality-of-life.
Love 100r! There aren't a ton of examples online, but their livecoding music software/language, ORCA, is a remarkable instrument.
https://100r.co/site/orca.html
Love orca, and that's a really nice example. Messed with it a bit when it came out, and one toy project I'll share in the hopes that someone does it before me: an orca GUI that uses a larger grid with representative images in place of single char glyphs. I found that writing orca is fairly straightforward — you look up the sheet, find a thing and do it. It's reading that's the hurdle. An 8 char chunk that made perfect sense when I wrote it takes just as many lookups to read later. This probably gets easier over time, but I still think it's a cool design opportunity.
I work with data warehouses, but I'm really jealous of the way our Finance team uses some abysmal plugin to directly query our GL from inside Excel - building something like that the can make the contents of a modern data warehouse available to Excel users has always been a holy grail for me.
My hunch is that exposing free-form SQL in Excel doesn't work, but something more like structured metrics (something roughly like dbt metrics) could potentially work? And tooling like this is probably what I'd want to prototype with.
Airbnb wrote a few great blog posts about why they built a standardized metrics layer - I think this one gives better technical context on the "what" and "why" than this announcement does:
In practice - complicated time series metrics, especially on top of derived temporal-logic attributes like funnels, activation, etc., are phenomenally difficult to write by hand in SQL for most analysts. We are in the process of switching to dbt metrics and it cuts the effort down for this kind of thing 5x-10x, and the SQL code dbt generates runs signficantly faster too.
One of Transform's strong suits is their Tableau integration - they have really good tooling for pushing metrics out to Tableau, rather than relying on Tableau to pull them in. Apparently AirBnb was/is still a very dedicated Tableau shop.
Minerva actually had really poor tableau support as of a year ago, that is something Transform improved on. Most of Minerva consumption was done via Superset, GoogleSheets, and email
The biggest win to me is: when your data pipelines are in SQL, changes and maintenance can be somebody else's problem.
I've had a ton of success asking our marketing and business teams to own changes and updates to their data - very often they know the data far better than any engineer would, and likewise they actually prefer to own the business logic and to be able to change it faster than engineering cycles would otherwise allow.
We do the same thing with our business logic & product configuration. I still haven't found something I couldn't expose as a SQL configuration opportunity.
Even complex things where you have to evaluate a rule for multiple entities can be covered with some clever functions/properties.
With some oversight this can be fine but non engineers can easily end up making the sql infinitely slower not to mention get things wrong (most commonly doing inner joins or have nulls in where in joins).
I've seen plenty of devs and da's do the same. The nice thing about SQL is it's easy enough to create a query that gives you what you want, but at scale it falls flat or purforms poorly. It's easy enough to work through most of the time, but too many lack the understanding of knowing when bottlenecks are likely to happen, and if/when it may be an issue.
I think of more than a couple basic joins in a query to be a code smell.
But in general - as an industry, we aren't there yet. All judgments about Meta's practices aside - no large organization likely has the ability to comply with a law like this without years or decades of engineering work, and the law seems pretty clearly intended for selective enforcement.
As small models continue to improve, and edge hardware becomes more capable, we would really like to run larger models that could incorporate full page content and screengrab data, which would be more likely to catch these kinds of attacks.
But we also find that sites that do one shady thing usually do others, which is a big reason why a tiny model like this can work - and why we are betting on low latency being a differentiating factor in real-world impacts.