We need a term for the dark pattern of zooming into just that part of the y-axis where the two closely competing benchmarks sit, to make the top one appear maximally better.
Spinning the present situation or the history of nuclear arms as a high-five, "go team human!" success story is... quite the take (Vasili Arkhipov, Cuban missile crisis generally, the current doomsday clock being "the closest the Clock has ever been to midnight in its history").
> You don't have help. But perhaps you could at least avoid discouraging people unnecessarily?
More germanely, you don't have to worry yourself, but at least avoid discouraging people with legitimate worries who want to take precautions. Even the present situation with nuclear weapons, precarious as it is, would likely be more precarious were it not for the political pressure of the people worrying in the 1950s and 1960s and up to today.
>Spinning the present situation or the history of nuclear arms as a high-five, "go team human!" success story is... quite the take (Vasili Arkhipov, Cuban missile crisis generally, the current doomsday clock being "the closest the Clock has ever been to midnight in its history").
I edited a response to this in my grandparent comment.
>More germanely, you don't have to worry yourself, but at least avoid discouraging people with legitimate worries who want to take precautions. Even the present situation with nuclear weapons, precarious as it is, would likely be more precarious were it not for the political pressure of the people worrying in the 1950s and 1960s and up to today.
Sorry, I may have mis-communicated. I want people to take precautions! That's why I linked to PauseAI: https://pauseai.info/ I'm trying to push back against defeatism.
When writing math papers, many (but unfortunately not all) mathematicians go through a post-processing step, where they take their ideas and proofs, and try to reduce them to simple and reusable core ideas that can be understood by the reader. Good writers will often also provide some representative examples that guided the proofs, explaining why various intermediate results can't be strengthened and why the proof can't be made much shorter without inventing new techniques. If AI-generated proofs were required to go through such a post-processing step before being published, that would go a long way towards improving the situation.
It's funny that seems like a step the human mathematicians would want, and (at least for now) might still outperform the machines on. In the same way that, eg, the notebooks of Galois contained the core breakthroughs in a messy form, and generations after him simplified and synthesized those ideas, until you finally have books and videos accessible to undergraduates.
One proposal that Tao hints at is to not rush to announce solutions. Instead maybe the AI companies should work privately with the subject matter experts on how to communicate the discoveries.
If frontier labs had chosen to go the path of offering to assist in existing endeavors, helping to build knowledge alongside researchers in ongoing projects and following ethical and professional research standards, we wouldn't be having this discussion at all; everyone would be stoked. Instead we have companies that disgracefully try to scoop researchers and fail to properly attribute earlier work and instead rebrand it as their own (what we normally call plagiarism) to make marketing material.
having labs open their research: what harness system they used, what types of problems they tackled, which problems success and which fail, how they success and fail so we have a better idea of what tasks LLM are currently good at
Failure to imagine an incentive doesn't mean there isn't one. You can't rely on this type of thinking to reason about security. The thing you would have never thought of is what gets you.
For example, an ad provider itself can be hacked by a malicious party, so the "pay money for" part no longer applies.
Or an attack by a state actor or other large entity, where paying for a coordinated disruption of some region or company makes financial or military sense.
Those are just two things that came to my mind, and are likely a fraction of plausible incentives someone might have now or in the future. People are creative and unpredictable. Weird shit happens. Fact is stranger than fiction...
Instead, just ask: should visiting a website ever have the power to freeze your computer without your consent? If you think the answer is no, this is a security bug and it should be fixed.
That’s why I said I’m sure it does exist but based on it only freezing the computer until you reboot it that’s not useful for hackers doing it to make money and doesn’t seem that useful for disruption unless of course you do hack a bigger ad network then I could see it legitimately being disruptive rather than a slight nuisance. It definitely should be fixed though, it’s crazy it’s gone so long with no fix
I'd guess GP id enraged that you can't selectively choose, or that there is no option to have the tabs restored without auto-loading, so you can keep your list of open work without being forced to reload a freezing tab.
On Firefox, when it reopens, it isn't actually running any tab other than the one you were looking at when it closed.
When you switch to one of the other tabs, that's the first time they are actually instantiated and run. So if you had this tab in the background, your machine crashed, and Firefox reopened on boot, it would not crash again.
Often, it will report "We are having trouble restoring your session", seemingly without reason, and that dialog has a very simple system for choosing which windows and tabs to recreate vs which to discard. I think you can change a setting to always use this dialog.
Since the machine is actually frozen, it enables many plausible social engineering scams ("We have detected a virus that froze your machine. Call this number for help..."), and I bet it has been used that way.
Not to mention possible data loss, interruption of work at a critical time, and so on.
And it doesn't foreclose the possibility of LLM achieving human-like writing ability. If I stole the writing of a human author much better than me it would still be plagiarism.
Of course, with real books we do have the problem of ghostwriting, where an author willingly writes for a book that will be published under someone else's authorship. That may be where things end up here, with requirements to acknowledge sources, whether ghostwritten for you or written by others.
Someone's actual writing (or talking) is a rich stream of information about who they are, their motives, their preferences, modes of persuasion, and so much more. Undetectable LLM writing essentially allows someone to assume another person's identity. That is not good for anyone except the person trying to pull off a "scam" of some sort, in the broadest sense of that term. It's bad for everyone else, and for society at large.
"Productivity" is not. It's only good for fake productivity. The consumers of the product will almost always be better off with you disclosing what the LLM did, and what you did.
reply