fucking is it? AI labs certainly want you to think so but they're running out of time and money. they've been telling me that the advancements of the last three months have made this a reality for the last five years. it's starting to feel a lot like Elon promising fully automated taxi services by next year... 11 years ago
For the completion to work, the source text needs to be ‘good’. That’s a basic kernel of how the thing works. Even with a perfect oracle autocomplete if the source text is ‘bullshit’, the output is too.
Or, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
So far, the LLMs I've used need problems to be fairly specifically scoped, or they don't produce the correct solution. Scoping problems correctly is a different skill-set than implementation, but it's still a technical skill that is expected from mid-level and higher engineers.
Yep and I just ran a simple enterprisey "ambiguity" bench on the big three (US) model providers: same ambiguous initial-prompt with same clarifications and pushback prompt sequence afterwards.
The edge of correct/better when facing ambiguity is very fuzzy, all models from the past 6 month or so have similar random ways of spinning between too-literal avenues and oddly misplaced misled fixations. Taking the right initiatives in face of uncertainty is definitely AGI, and its not there, and perceptrons + attention layers just ain't got what it takes no matter how hard you push.
I don't think pg is giving advice on what will lead most directly to a job, but rather what is the best learning for a 17yo.
A 17yo who trains their own LLM will have a much richer understanding of what AI is, how it works, what its potential capabilities and pitfalls are, versus someone who spends the same time doing something else.
Quality and speed are not diametrically opposed. A great engineer does well on both axes by building the minimal thing needed now in a way that is easy to extend in the future.
I have also seen projects go badly because the eng was trying to be perfect upfront. Whereas quickly getting to an MVP and then iterating tends to go better.
I’m curious about this scale vs speed distinction.
Every codebase includes parts that are more experimental, and parts that are more core. My sense is that AI can help on both of these fronts (I.e building rapid prototypes on the fringes and hardening the core with better test coverage).
Engineers seem to think business people don’t know what they are doing, but if your post were true, then companies would add slack to outperform their competitors.
The broken system likely doesn’t have enough business impact to justify the investment to maintain it.
When you plan working 3-5 years in a single company you don’t care if it crashes and burns month after you leave just to burn down next one.
Conversely we see the same dynamic with engineers, they build stuff to prop up their CV and don't care if company still supports crap they did after they leave.
It's a measurement problem, which engineers also fall prey to, perhaps even more.
It's the danger of data driven decision making. Cutting people and resources right now gets you a measurable gain. Not cutting them gets you a gain tomorrow.
But, that gain is unmeasurable! Because in order to measure it you would need to know what happens in an alternate universe where you cut those people. So, if you're only making data driven decisions, you would cut the people 100% of the time.
But that's why companies aren't run by algorithms, they're run by people. The algorithm would run the company into the ground.
> companies would add slack to outperform their competitors.
I think if they did this they'd get buried by the market. Your slack is someone else's opportunity to undercut you. It's a systemic problem, it's in every individual's self interest to work towards instability.
> When his supervisor sent him a paper to read, Bob asked the agent to summarize it. When he needed to understand a new statistical method, he asked the agent to explain it. When his Python code broke, the agent debugged it. When the agent's fix introduced a new bug, it debugged that too. When it came time to write the paper, the agent wrote it. Bob's weekly updates to his supervisor were indistinguishable from Alice's.
In my experience, doing these things with the right intentions can actually improve understanding faster than not using them. When studying physics I would sometimes get stuck on small details - e.g. what algebraic rule was used to get from Eq 2.1 to 2.2? what happens if this was d^2 instead of d^3 etc. Textbooks don't have space to answer all these small questions, but LLMs can, and help the student continue making progress.
Also, it seems hard to imagine that Alice and Bob's weekly updates would be indistinguishable if Bob didn't actually understand what he was working on.
Faster doesn't always mean better. I've "learned" things from LLM really fast, but I don't retain the information the same way as if I had taken my time to really work through it
My prediction is that we'll start to see a whole new layer of abstraction to help us write high quality code with LLMs - meaning new programming languages, new toolchains, stricter typechecking, in-built feedback loops etc.
The slop we're seeing today comes primarily from the fact that LLMs are writing code with tools meant for human users.
