LLMs are fine-tuned through human feedback. One way to improve for them would be to become more helpful, more factually correct, more capable of solving problems.
This may be (much) harder to do than generating rhetoric that is convincing to a large majority of people, perhaps so convincing that it can be classified as a superstimulus that bypasses any critical thinking in those that are especially vulnerable. It might not require anything like general intelligence.
If this is so, then it is the path they are going to take, not out of some malicious plan but as a simple matter of statistical probability. It's well known (especially in "AI alignment" circles) that any metric that can be gamed, will be. Falling over really fast vs. learning to walk, etc.
It feels to me like this is happening, and that many of those most exposed to LLM output display a literal inability, like some kind of blind spot, to notice the most blatant errors, and a complete conviction that those things have actual intelligence and even conciousness. And trying to argue them out of it can be like explaining the Monty Hall problem, some are just incapable of getting it.
And many of them have lots of money and power. IMO, that's the real risk, rather than sci-fi scenarios about perfectly simulating someone's brain in order to convince them to let the AI out so it can turn everything into paperclips. To do real damage, it only needs to be persuasive to some powerful people, most of the time. It does not need to have conscious intention, or planning, or even the rudiments of what one might call general intelligence. Just blind brute-force optimization for generating convincing bullshit.
Maybe the victor will be whoever doesn't fall into the trap of believing that a next word predictor optimized through RLHF to sound convincing to the average human [1] is in any way intelligent, and restructuring their entire economy and military around doing whatever the magic oracle machine says.
If the entire "free world" goes all-in, the history books will likely be written by Iran, Russia and North Korea.
[1] possibly to the point of presenting a superstimulus that bypasses any facilities for critical thinking. This might be easier to solve than many other problems that are used for evaluating A"I" performance, and very likely doesn't require genuine intelligence, just brute-force search + evolution
Yes, and it's not quite the same as a normal "INT 01h". It causes a debug exception, which may enter ICE mode if it is enabled (undocumented bit in DR7, or PMCR on Pentium), otherwise it invokes interrupt 1, but without checking the privilege level on the IDT entry, or the interrupt redirection bitmap in V86 mode.
IIRC, older versions of the Linux kernel had a security bug because they didn't expect this to happen.
ICE mode was sort of a precursor to SMM, but both also coexisted for a time with slightly different behaviour. It was introduced in the 286, where instead of ICEBP there was "STOREALL" (opcode 0F04). F1 on that processor was a prefix instead, with the same function as UMOV on 386+. If you use them together - something Intel probably didn't intend - you can dump the internal CPU state to memory on a regular non-bond-out chip.
If you don't know anything about the environment the code runs in, you can't rely on what action the undefined opcode handler will take either. (same for any other exception)
1. you can use the opcode, it's guaranteed. if it does something else, that's not on you.
2. you might know everything about the environments and that's how you know that different environments use different interrupt tables, but you want a code snippet that will not rely on that, so you use this guaranteed opcode.
Usually, you do know something about the environment that your code runs in. But in the case that you don't, causing an exception (whether "undefined opcode" or anything else) can't be guaranteed to terminate the process, because some operating systems actually did use it for other purposes. The assembly language equivalent of nasal demons :)
An interrupt or SYSCALL instruction could do anything, which includes remapping or overwriting the memory location it returns to. So no, these instructions can't be prefetched in any case.
#UD has the same stack frame as a software interrupt, there's no error code pushed. But most likely, executing INT 06 from ring 3 will generate a protection fault instead, since the gate descriptor would be set up to not be reachable from that privilege level.
(exceptions that do push an error code couldn't be emulated at all using INT, since the error code is the last thing pushed by the CPU, after flags and return address)
This looks like slop, it's all concatenated into a single file and most probably not based on the actual malware. I'm fairly sure that for example the real one does not include the literal string "Stuxnet" anywhere, like it does here:
The original name given by VirusBlokAda was "Rootkit.Tmphider;"[41]
Symantec, however, called it "W32.Temphid", later changing it to "W32.Stuxnet".[42]
Its current name is derived from a combination of keywords found in the software (".stub" and "mrxnet.sys").[43][44]
And the Wikipedia article explains where the name came from, a combination of ".stub" and "mrxnet.sys".
Not one literal string as it appears several times in this purported "reconstruction". Including as the name for a registry key, in the hex code at the end of an EXE header stub ("REALTEK",0x00,"Stuxnet"), and in a frigging autorun.inf as the program name.
Even if Wikipedia is wrong and that string should appear somewhere in the original binary, whatever LLM they used has really been overdoing it beyond the bounds of realism: "Hey look, it's the REAL STUXNET, you've all read about it, here is the 100% real authentic reverse-engineered source code!"
Mentioned in the article, same link too. The court documents included a diagram of the microcode format, and also a comparison of the reset sequence between Intel and NEC.
Generally speaking, Linux doesn't suffer from ELF-executable-file-infecting viruses in practice, because most random executables aren't run with root permissions, so they don't have permission to write to widely-executed executables, which you install from the package system.
However, anything in the build toolchain that someone uses to build those widely-executed executables necessarily does have permission to write to them!
This is certainly not novel, having been the subject of a Turing Award speech, but it is something that many people haven't thought through.
This may be (much) harder to do than generating rhetoric that is convincing to a large majority of people, perhaps so convincing that it can be classified as a superstimulus that bypasses any critical thinking in those that are especially vulnerable. It might not require anything like general intelligence.
If this is so, then it is the path they are going to take, not out of some malicious plan but as a simple matter of statistical probability. It's well known (especially in "AI alignment" circles) that any metric that can be gamed, will be. Falling over really fast vs. learning to walk, etc.
It feels to me like this is happening, and that many of those most exposed to LLM output display a literal inability, like some kind of blind spot, to notice the most blatant errors, and a complete conviction that those things have actual intelligence and even conciousness. And trying to argue them out of it can be like explaining the Monty Hall problem, some are just incapable of getting it.
And many of them have lots of money and power. IMO, that's the real risk, rather than sci-fi scenarios about perfectly simulating someone's brain in order to convince them to let the AI out so it can turn everything into paperclips. To do real damage, it only needs to be persuasive to some powerful people, most of the time. It does not need to have conscious intention, or planning, or even the rudiments of what one might call general intelligence. Just blind brute-force optimization for generating convincing bullshit.
reply