I wonder if there is a principle here we can apply to neural nets like dividing up the layers for different tasks (no not MoE but something more fundamental).
Tbh I can't see how any site about tooling doesn't go the way of Stack Overflow. I used to follow the latest framework and CSS articles avidly. With agents it's a moot point.
It's sad. But rather than fight it, we should be thinking about what communities we form next, in a world where knowing flexbox tricks isn't a flex.
That's because those AI's were trained on existing "CSS tricks."
If people stop writing new CSS tricks, they will never make it into any dataset, and it won't matter if the AI can search the internet or not, because they won't be posted on the internet either.
Oddly enough if the scraping is good, AI can find the CSS tricks in the wild. Not as applicable with other technologies but scrapable web is readable for AI much like how some of us old people learned html/css by looking at source of quality websites.
I think this assumes AI needs the same kinds of “tricks” humans do.
Flexbox, CSS Grid, etc. are mostly abstractions that make low-level DOM/layout control tractable for humans. But from first principles, an agent doesn’t really care whether the interface is elegant. If it can reliably manipulate the underlying system directly and verify the rendered output, it can just write whatever low-level code gets the pixels where they need to go.
So I don’t really buy that there’s some big missing abstraction layer left to invent for AI here. Once the agent can control and verify the underlying output, better human-facing wrappers mostly stop mattering.
Not talking about design trends, just the mechanics of producing the UI.
What low level code would that be? Position every element manually with JS and position: absolute? Flex and grid are the low level layout mechanics used for any responsive webapp. Even most native frameworks use the flexbox model.
no you still probably use layout algos but most variation on top of them in css is human preference v capability (Extra HTML Elements v Pseudo Elements etc).
Cost will keep dropping, but the frontier will keep getting pushed. The whole "give me today's model 10x cheaper and I'm good" line is a fallacy. It isn't true now and it never will be for the top 1% of tasks, which will create the most economic gains.
> It isn't true now and it never will be for the top 1% of tasks, which will create the most economic gains
Vulcan Materials Company has produced some of the strongest and most resilient economic gains for investors at around 25% gross margins for 50 years.
Vulcan’s business is crushing rocks, and then driving those rocks to where people need crushed rocks.
I mention it because Vulcan isn’t a sophisticated business at its core, but its economic returns are exceptional, because they do their unsophisticated work exceptionally well, and exceptionally efficiently.
Across the economy, most economic gains are created by companies like Vulcan, who do boring repetitive work exceptionally well and efficiently.
I expect this to hold true into the era of AI. Most stuff probably doesn’t need an exceptional model, and paying for an exceptional model to do unsophisticated work will leave your business vulnerable to competitors who take time to find the most efficient model for the task, and undercut you.
I just wonder whether the analogy holds from atom based businesses to byte based businesses. At the end of the day, there are pretty hard constraints on how much a competitor could undercut Vulcan - they'd need to be better a) crushing rocks and b) driving those rocks to where people need it. Something about that, while simple, seems really nonfungible to do well. The business may traffic in commodities, but the service and quality it provides ironically seems to be not really a commodity.
Conversely, AI models feel pretty different. People will swap models at the drop of a hat and while not perhaps fully fungible, the cost of switching can be as low as one engineers it to be. In this case, the service does really seem to feel more like a commodity.
Not sure where this leads or how this ends. Perhaps those two will swap poles for me (IE as robotic automation improves, there are more commoditized Vulcans, and as metered intelligence improves, they become less fungible). I just can't quite see how yet. Just food for thought.
There are an enormous number of tasks that can get by on good enough.
If you need image recognition, and a 30B model saturates the use case with 100% accuracy, you absolutely wouldn't continue to use the next frontier model as they come out.
And I'd argue most economically meaningful tasks will be saturated by cheaper models than those requiring frontier.
Think about what today's models can do with pretty close to 100% accuracy, and then consider that they will be orders of magnitudes cheaper over the years.
5.6 Sol can already obviate tons of labor, and why would you pay 2x or more for no meaningful gain?
The relative gap between frontier and non frontier also continues to shrink, so it's not like you take a meaningful performance loss by rewinding to models from 3-6 months ago. And soon that gap will expand to 12-24 months.
I get the impression the majority of people on here only think about coding, which net net will be a tiny volume of overall AI use in the end.
That can all be true but the frontier models will still have a huge market. You're thinking of all tasks as a fixed pie. The top 1% of intelligence opens up a whole new pie, stuff nobody does today because it's too expensive: daily cancer scans instead of one every few years, asteroid mining missions that need ten thousand PhD-hours of planning, custom drugs designed for your specific tumor, a personal lawyer and doctor for every person on earth, auditing every line of code in every bank and hospital continuously and so on.
> daily cancer scans instead of one every few years
Silly nitpick: the reason we don’t do daily cancer scans isn’t the cost, it’s the false positive rate. Invasive procedures like biopsies come with complications like infections that happen at a higher rate and do more damage than the cancer that doesn’t even exist. This dilemma is pervasive in medicine, because our tests aren’t perfect but the thing they’re testing for is rare.
The whole paradigm changes, though, when you can do daily cancer scans. You don't get a biopsy when the scan shows a lump. You get a biopsy after a couple weeks of daily scans showing the lump growing. Plus, having all the data from the daily scans improves your testing accuracy so false positives and negatives are more rare.
The errors are correlated, not random. Lumps are usually benign cysts. If you start cutting people open for every cyst on a scan, you’d kill a lot more people than you’d save.
This isn’t something you can solve with more scans because the tests test for data that is indistinguishable. They look the same on a scan, there’s an overlap in the assay with some random protein with the same binding sites that is only present in 1% of the population, the coding gene in one person gets repeated in a noncoding region in another, and so on. The “more data” that works is a doctor applying professional judgement (which they’re also famously bad at because biology is a fickle mistress).
If you're doing non-redundant tests and your uncertainty bars aren't shrinking, it's usually a skill issue.
If it looks like a duck, it might be a duck - or a painting of one. If it looks like a duck, swims like a duck, and quacks like a duck? The joint duck estimation is much more confident now. There might be a few more observational tests one should administer before committing to a duckhood decision, but each tests pins down variables and rejects confounders. Uncertainties are cut down, and we get closer to crossing the threshold between "duck-informative" and "duck-actionable".
Thus, it's often worth it to improve observability. If you managed to make a certain test more reliable, or cheaper to administer, or reduced the chance of adverse effects? Or, in other words, improved SNR, reduced costs, and reduced costs? You can get more information for your buck. Paired with good knowledge: you can make better decisions more easily.
The fact that the thought of "having more information might be bad actually" even occurs in the field of medicine shows just how far it is from being optimal. Having more information isn't always beneficial - some information is genuinely redundant. Some information is not worth the effort of gathering and integrating it. But if you get more information and it results in worse outcomes? You're doing something wrong.
More data (daily scans) can mean we get better at medicine, though, and more accurate. You're assuming "daily cancer scans" look just like they do today, rather than eg unobtrusive devices in our daily lives that measure changes over time.
The frequent "muh false positives" comment we hear from doctors appears to be a lack of imagination?
Yes, agree that token consumption will increase exponentially for the next while.
Disagree that the frontier model is where the economic gains will be realized.
The smaller the relative gap between frontier and non-frontier/open weights, the less pricing power.
This gap has shown only to shrink over time, not expand.
Businesses will pay more for frontier, but not meaningfully more to justify the economics. It's always going to be a low margin business, perhaps outside of cyber security, warfare/intelligence and perhaps drug discovery.
Though the expensive and time consuming part of drugs is doing the trials and getting approval, not coming up with ideas
Sounds kind of like another K shaped economy. Low end models will be highly competitive and low profit. Problems that can be solved by low end models will be highly competitive and low profit too.
Where the interesting work will be is at the median point where cheap models do almost all of it but need to hand off some parts to the SOTA/more expensive models. Seems like there's money to be made by maximizing low end use while maintaining quality.
Certainly there's still a business there, I'm not saying they won't exist. But it's not going to be a monopoly-esque business with so many players in the ring, OpenAI, Anthropic, Google, Meta, Deepseek, Alibaba, GLM, Kimi etc. It will be cutthroat and a race to the bottom on price. And the difference from today -> 6 months ago intelligence will not be very meaningful.
Investors are largely treating these as future monopolies though.
We can already do so much with existing models. Harness improvements are probably more meaningful at this point.
e.g. say most image recognition can get saturated by a model of size xB parameters, so your tool for that can handoff to a smaller model. Document text extraction can use a model of size yB parameters. A model of size zB for summarizing text.
We are starting to get to a point where you can reasonably scope out an upper bound of required size/effort for many common tasks, and if you string these together, the frontier will largely act as an intelligent invoker of more efficient models.
Up until now there have been meaningful gains to each of those types of workstreams by using newer models, but that is starting to no longer be the case.
Yes, I do believe token consumption will rise exponentially from here in the near term. But cost of switching is low, and substantial profitability will be difficult.
how much would you pay for a prompt that could cure cancer? if you're a pharma company you would pay millions to get there days faster than your competitor. as intelligence rises the marginal value it can deliver rises with it.
Something like curing cancer (more realistically, curing a specific kind of cancer) has to interact with much slower real-world processes. The most expensive part of drug development is Phase 3 clinical trials in humans. Even the smartest model in the world can't accelerate that meaningfully. Even much earlier when drugs are just testing in cell cultures, it's a lot slower to run lab tests than to run software tests or mathematical proof checkers.
Or to put it another way, there's enough natural variation in real-world bottlenecks that no pharma company can assume they'll beat competitors to market by using a smarter model.
A really smart model could significantly improve the pharma business if it could identify promising approaches to cancer treatment that are less likely to fail in clinical trials, but I don't think that the frontier labs have data to make that work yet. Much of the biomedical literature is poorly reproducible ("replication crisis") and much of the drug-development-specific data is proprietary, never published in the first place.
I do have hopes that general laboratory automation will go faster with LLM assistance, even if all the LLM does is write Python glue scripts to enable custom workflows and instrument integrations.
Is this sarcasm? I assume most studios are integrating AI into their workflows, but I still haven't seen a single vibecoded game that looks interesting.
I think there's an intelligence limit, or at least asymptote. It may be above human intelligence, but I don't think it's miles above it (at least not the kind of intelligence humans can create, recognize, or use). For example in Go, most estimates place God or "perfect play" three ranks above top professionals[0]. In the latest human-AI Go match, the human got a 2 stone handicap. So it's not like we have a lot more frontier to push there.
Just FTR, Shin Jinseo's result in that latest match is seen as extraordinary. In the last few years it's been common for top pros to play bots on 3 or even 4 stones.
The thing about playing strength in games like go is that it doesn't map at all linearly to results. It could very well be the case that the bots are only a few points of handicap away from perfect play, but thousands of points of ELO.
Anyway, this is a poor argument for a possible "limit to intelligence" because of course the game is finite (even if the search space is extraordinarily large). Of course you can't beat the "hand of god", by definition. Of course you can create games and puzzles for which "hand of god" is a coherent concept. It shouldn't be too hard to specify games where it isn't, though (perhaps something involving real numbers).
Intelligence is spiky. In some things humans may play near the limit (Go possibly), but if you look at parts of mathematics like addition, humans can add in their head just fine but its a few trillion times faster to use a computer on addition problems of any size.
And that's not even really touching societal/network intelligence. A single human isn't that smart and can't accomplish that much. Hence we form families, and companies, and societies, and governments. What does a society of AIs look like?
Note that addition that's "trillion times faster" is purely an optimization, not greater intelligence.
And a society of humans does not increase our overall intelligence. It allows all of humanity to access the accomplishments of our most intelligent members throughout history (which is huge, don't get me wrong!) but it seems almost self-evident that our civilization as a whole is not smarter than Newton/Einstein/von Neumann/etc.
Intelligence is also access to data, along with all of it's other requirements. Society as a whole is far smarter than Einstein.
If I dump Einstein on an island at 5 you don't get a theory of relativity. The data provided by a civilization is required, hence why people 10,000 years ago didn't go to the stars. It is unlikely they were significantly less intelligent than us. They just didn't have the language of data we do now.
this assumes our whole universe and what we can do in it is a finite go board. maybe it is. but we are no where close to exploring even a fraction of it. lots of things need to happen before any limit is reached. the cavemen would also probably think we saturated tools once they saw bows and arrows.
Intuitively I agree with you, but you've cited evidence that doesn't actually prove what you said. Yes, openrouter users choose cheaper open weights. If you're on openrouter, you're already shopping around. We'd need stats for claude, codex, gemini, grok (yuck), AND openrouter, to determine what people in general (not just the subset of people who decided to shop around) are actually doing.
This is obvious in hindsight. I expect most companies that move fast and care about performance to abandon React Native. Code is cheap now, and the tradeoff of having a single codebase and only hiring React devs basically isn't there anymore.
No that's not how the world works. Most of climate change happens not because some decrepit old man lives on his yacht but poor people in third world countries get their first air conditioners. As they should.
The only way is through. Most of are not going to be living the suffocating life you think we should. We will grow, innovate and accelerate out of this.
> Most of climate change happens not because some decrepit old man lives on his yacht but poor people in third world countries get their first air conditioners.
You're confusing acceleration with velocity. Poor people are far from the greatest per capita contributors.
If everyone had devices that could do some compute worth paying for, people would be doing the work all of the time. The problem is actually useful POW is a lot more expensive compute-wise, making it non viable for normal users
I think in the future we'll have robot pets and look back in horror at how we enslaved animals out of their natural habitats and normalized (even glorified) it.
What about the animals that have been domesticated to be pets? It's arguable that their native habitat is the home at this point. Although, I do hope we stop capturing the animals which aren't domesticated (most "exotic" pets).
I agree with the horror of pulling animals out of their natural habitat, but if you are using this as a blanket statement towards all living animals then you should read up on the history of the domestication of wolves (dogs). It was born of a beneficial relationship to both humans and wolves and i personally find my relationship with my dogs to be extremely fulfilling and i believe my dogs do as well.
I have no interest in a relationship with a robot.
I know the history. We bred them to be our pets and normalized this relationship. Your pet might even love his life. But we can do the same with robot AIs, and it will be a better relationship. Your robot can talk to you. It can call 911 if you’re in trouble instead of just barking.
There are many many people for whom that will never be close to the same, let alone better. An immortal sack of wires that is passed down through generations is interesting and will appeal to many, but it’s not automatically a “better” relationship than a living creature.
> Your robot can talk to you.
If you want a conversation, may I suggest another person instead of a pet?
> It can call 911 if you’re in trouble instead of just barking.
If you want a way to automatically call 911, how about a smartwatch that tracks your vitals and does just that?
Anyway, stories of pets calling 911 are a dime a dozen.
Where would the dogs go? They don't have their own ecological niche and usually can't survive in the wild. In cases when they actually can survive in the wild, stray dogs are known to wreak absolute havoc on the natural native ecosystem. Total dog extermination?
I'd argue that emotional co-regulation is born out of biological origins. Unless we are making synth, it would be a long time to match the relationship with a living creature.
> I have no interest in a relationship with a robot.
I think I could, if it meant I could keep the experience of having a pet.
I've had cats going back to my twenties. I dearly love my fifteen y/o cat. She's my last cat, though. As I approach fifty I've grown tired of hefting bags of litter and food, stooping to clean up after her, taking her to the vet, and administering her medication.
When she's finally dead (which is hopefully not for a good long time) I'm not going to get another. This makes me really sad. I'll miss having a cat around. As I age, though, I don't want the encumbrance of taking care of a pet, nor do I want to cause hardship to family if I suddenly become unable to care for a pet (dying, being hospitalized, living arrangement changes and owning a pet isn't allowed, etc).
If there was a robot alternative I'd jump on it. I think my brain would adjust fine to a convincingly-executed robotic cat. Our brains instinctively ascribe agency to objects moving in non-inertial frames, for example. Think about all the people who name their Roomba, install googly eyes on it, etc. People easily accept inanimate objects that behave like living things, to the point of developing feelings for them.
When my cat is gone I hope to get the opportunity to choose a "relationship" with a convincing robotic cat versus not having one at all.
I wouldn't expect everyone to feel this way. I certainly don't want to legislate away pet ownership.
I bet that, if good robotic pets become available, we'll see a significant number of people favoring them over living pets for convenience, long-term cost, freedom from anxiety about the pet's health and eventual death, and the opportunity for companionship in living arrangements where pets aren't allowed.
I've said, for years, that I think there's hundreds of billions of dollars for the company who can make really good robotic pets. If somebody can get the "soft robotics" down and build / train good animal behavior models I think they'll be able to print money.
Yes, and robot pets aren’t going to look like clankers. They’ll be warm and furry and imitate the sounds and movements of real pets. I think people still imagine robots as some cold metallic insert-Stephen-Hawking-voice thing, and it’s pretty clear even with today’s tech that’s just not what they’re going to be.
Interesting how all of the replies so far to your comment assume we’d just release the existing animals into the wild instead of just not breeding more.
As one might guess, the "just" there is doing a lot of work.
Common pets reproduce just fine without our intervention - in practice it requires costly maintenance programs just to keep their population under control (not to mention the massive and ongoing eugenics program required to produce only the desirable breeds/attributes).
If your neighbourhood isn't overrun with stray cats and dogs, that's because someone is funding a spay/neuter program, and someone is funding animal control to round up (and ultimately euthanise) the rest.
All animal husbandry systems have to be maintained via regular culling, and "eliminating" pets would require orders of magnitude more killing
I always assumed the "enslaved animal-free" future would be something that came on gradually, likely motivated by economics (i.e. lab-grown meat becoming cheaper and higher quality than farmed, robotic pets being cheaper and more "trouble-free", etc).
I never felt like anybody seriously thought we'd release domestic animals into the wild to fend for themselves-- that's ludicrous.
> look back in horror at how we enslaved animals out of their natural habitats
Somehow, vegan-inclined folks never seem to factor in the mass-genocide of domestic breeds that is necessary to achieve their particular utopia. We're talking 1.5 billion pets, 5 billion livestock, and 28 billion poultry - none of which are remotely able to survive on their own in the wild.
Wait... do you really think that if everyone goes vegan these animals will get killed, but if everyone eats them they won't?
You are aware that all of these livestock animals didn't exist a few months to years ago? And that we go through these animals every couple months-years?
Everyone going vegan would just mean we'd stop repleneshing them.
Keep in mind that we already have to perform widespread culling of wild animals in this category. Having eliminated most of the apex predators, we're on the hook to keep populations of wild horses/cattle/deer/etc in check.
If we cease to use domestic animals, they end up in the same boat - we're no longer paying to feed and house them, and if they are capable of breeding in the wild, they will breed unchecked. Leading us to cull them, ad infinitum
The "genocide" that you describe is happening constantly. None of the chicken we have today will be alive in a couple months. We kill them to eat them.
If we cease to "use" them, we just keep that "genocide" going for a year, but we just stop breeding new ones. The result is no more animals after a few months/years. And you didn't kill any more than you would have anyways.
If you have a sink that is half full because the inflowing water and the outflowing water are in quilibrium, you drain it by turning off the tap (breeding).
> we just keep that "genocide" going for a year, but we just stop breeding new ones
"Carry on for a year and then it's all good" is not one of the available choices.
Your choices are (a) wipe out the species entirely, or (b) allow them to continue breeding on their own, and commit to culling them for the rest of eternity.
I detest the vegan movement. I'm not an animal lover. Pets are unsanitary nuisances. Robots are better. We don't need to commit a genocide against domestic pets for the transition. Just breed less.
This isn't great but a negative side effect of an otherwise positive change. I know a lot of comments will try to make this into a gotcha but its really not. Decreased friction leads to misuse. Just like before we'll figure out ways (reputation, light filter pass) to deal with it.
Tech people kind of do, we just dont notice it until trying to describe things to others.
I had some YouTube video on aanf they were benchmarking something, my wife was like "what does any of that mean?"
I realised that, yeah that last sentence was basically just nothing but abbreviations and acronyms. I understood it perfectly but from the outside it is total nonsense.
This isn't just a tech thing, this goes for almost all specialtys.
reply