Transformer is a patterning probabilistic machine for a sequence of identities[1]. These identities are fed to the transformer in lanes. The transformer is conditioned to shift lanes one position to the left until they make it to the output, and make a prediction in the right-most lane that got freed up. Attention adds an exponential amount of layer interconnectivity, when we compare it with a simple densely connected layers. The attention mask serves as a high-dimensional dropout, without which it would be extremely easy for the Transformer to simply repeat the inputs (and then fail to generalize when making the prediction). Each layer up until the vertical middle of the Transformer works with a higher contextual representation than the previous one, and this is again being unwound back to lower contexts from the middle layer back to the original identities (integers) on the outputs. This means that you have raw identities on the input and output which span a certain width/window of the input sequence, but in comparison the middle-most layer has a sequence of high level contexts spanning extreme lengths of the original input sequence, knowledge-wise. [1]It's important to know that modification (learning) by the Transformer, of the vector embeddings which represent the input/output identities/integers that the Transformer works with, constitute big portion of the Transformer's power, and the practical implication of that is that it's impractical to try to tell the Transformer that e.g. some of our identities are similar or there's some logical system in their similarity, because all the Transformer really cares about is the occurrence of these identities in the sequence we train the Transformer on, and the Transformer will figure out the similarities or any kind of logic in the sequence by itself.
Life (aging) is a direct cause of cancer. And we can't change that unless we have some sort of nanobots constantly repairing DNA damage in every single cell of our body.
You realize the human body has a process thst cleans out damaged and misbehaving cells. Autophagy. This process is kicked off during periods of not eating.
We both expose ourselves to too Many toxins, eat too often, and don't get the right balance of nutrients over the long term.
If you can positively impact these things, then you can lengthen health span and lifespan.
Yes and no. While some cells may be impossible to rejuvenate otherwise, there are few reasons why we cannot just exchange many organs for dish grown ones, i.e artificially grown kidneys. For others like blood stem cells, we many extinguish the existing ones and add replace them by fresh ones that lack any pathogenic somatic mutations every so often. Even parts of the brain may be replaced one by one, though I assume such a technology is quite far away (compared to the other two that may be possible in the next 2-3 decades).
How's that related to this story? One outcome might have multiple factors affecting its probability of ocurrence. Just because you can't change life doesn't mean you can't stop alcohol. Life isn't 1 and 0 dude.
I can imagine aging being a technically solvable problem and maybe some of us will live to see it.
But if people didn't age, then there would still be a base rate of mortality from illness and accidents, and so for any arbitrary probability (99.9%, etc) there would be a maximum age that one could reasonably expect to live to.
For instance, if the only thing people ever died of was car accidents, at a rate of 20 per 100,000 people per year, then it would still be very unlikely for anyone to live more than 20,000 years. That's a long time, but it's not millions or billions of years, much less eternity.
Furthermore, if nothing but accidents killed people we might become extremely conservative and do practically nothing for thousands of years. Do I want to use the stairs and risk 99% of my potential lifespan?
So maybe one could say that biology and even physics don't ensure death, probability does.
I know I'd never be able (or wanting) to live with having left my dogs behind. I don't fear my own death _that_ much. I'm here for them until they cross, only after that I can do whatever I want with my life.
Damn, so I really need to follow up on my plans on moving to e.g. Thailand (and getting happy by following my dreams in terms of work/endeavors).
I don't have IBS but more like a colitis and I was really good for months but it came back strong after one of my dogs passed away and I got terribly sad.
What I'm getting at is that more than a vitamin D, I suspect a strong link to happiness and life satisfaction.
1. When people are financially incentivized to run servers, they always will.
2. If a crypto protocol doesn't evolve at the pace of available innovation, that particular blockchain will be superseded by a new one. That said, a (truly democratic) evolutionary process is a core part of every blockchain specification.
3. You can get blockchain data via public (and federated/proxied) API, but you can always cryptographically verify its veracity, and your edge device (e.g. your smartphone) can do that. The same the other way around, you cryptographically sign the inputs you send to the networks, so that no federated API can tamper them, because the secret key stays on your device. This is referred to as the "trust-less model".
It's interesting to see how human understanding differs when it comes to complex, yet clearly defined topics, like machine-learning/Transformers.
For comparison, my understanding of Transformers, after going through Peter Bloem's "Transformers from scratch" [1], implementing/understanding the code and the actual flow of the mathematical quantities, my understanding is that:
- Transformers consist of 3 main parts: 1. Encoders/Decoders (I/O conversion), 2. Self-attention (Indexing), 3. Feed-forward trainable network (Memory).
- The Feed-forward is the most simple kind of (an input->single-layer) neural net, actually often implemented by a Conv1d layer, which is a simple matrix multiply plus a bias and activation.
- The most interesting part is the Multi-head self-attention, which I understand as [2] a randomly-initialized multi-dimensional indexing system where different heads focus on different variations of the indexed token instance (token = initially e.g. a word or a part of a word) with respect to its containing sequence/context. Such encoded token instance contains information about all other tokens of the input sequence = a.k.a. self-attention, and these tokens vary based on how the given "attention head" was (randomly) initialized.
The part that really hits you is when you understand that for a Transformer, a token is not unique only due to its content/identity (and due to all other tokens in the given context/sentence), but also due to its position in the context -- e.g. to the Transformer, the word "the" at the first position is a completely different word to the word "the" on e.g. the second position (even if the rest of the context would be the same). (Which is obviously a massive waste of space if you think about it, but at the same time, at the moment, the only/best way of doing it, because it moves a massive amount of processing from inference time to the training time - which is what our current von-Neumann hardware architectures require.)
reply