https://www.youtube.com/watch?v=aircAruvnKk&ab_channel=3Blue1Brown ^ 3b1b vid on Neural Networks
https://www.youtube.com/watch?v=KNAWp2S3w94&list=PLQY2H8rRoyvwWuPiWnuTDBHe7I0fMSsfO&ab_channel=TensorFlow ^ TensorFlow YouTube -> Playlists -> ZeroToHero
https://www.youtube.com/@YannicKilcher/about ^ Good ML community formed around this YouTuber (links to Discord)
http://d2l.ai/ ^ Up-To-Date ML book/resource, uses PyTorch (got reco from above Discord)
https://www.simplilearn.com/keras-vs-tensorflow-vs-pytorch-article ^ Keras / PyTorch / TensorFlow (Theano dying)
CodeFusion: https://arxiv.org/abs/2310.17680v1 -- Applying SD + Transformers to code-gen (v3 has no paper)
Sits on: https://arxiv.org/abs/2212.11685 "Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise"
https://www.youtube.com/watch?v=THIIk7LR9_8&ab_channel=MITCBMM "Attention Approximates Sparse Distributed Memory " (https://arxiv.org/abs/2111.05498)
https://arxiv.org/abs/2310.08866 Adaptivity and Modularity for Efficient Generalization Over Task Complexity
-> https://arxiv.org/abs/2302.11529 Deep Modular Learning -- ?
https://www.rwkv.com/ RNN Transformer-alternative that seems to generate GPTx-level responses fast and cheap
https://arxiv.org/abs/2307.03170 Focused Transformer: Contrastive Training for Context Scaling -- seems to be using a memory-lookup to supply the weights in an attention layer(?)
https://arxiv.org/abs/2310.00194 A Prefrontal Cortex-inspired Architecture for Planning in Large Language Models
https://www.youtube.com/@Mutual_Information ^ Course on RL here
https://www.youtube.com/@deeplizard/videos ^ (tx Kake)
https://hackmd.io/@machine-learning/blog-post-cnnumpy-fast#III-Performance-of-fast-implementation ^ Amazing ConvNet-in-numpy
https://www.youtube.com/watch?v=J4Wdy0Wc_xQ&ab_channel=StatQuestwithJoshStarmer Beautful simple explanations from this YouTuber
https://www.youtube.com/watch?v=GXhBEj1ZtE8&ab_channel=AlexFoo 3b1b-level exposé
TODO: How do Hinton's Capsule Networks relate to xformers? https://analyticsindiamag.com/what-happened-with-capsule-neural-networks/
First I watched https://www.youtube.com/watch?v=rVzDRfO2sgs&ab_channel=EdanMeyer (40min) which draws out some visuals.
Then I listened to https://www.youtube.com/watch?v=NWqy_b1OvwQ&ab_channel=EyeonAI (which unfortunately has none).
Then I downloaded the transcript and picked through it, googling a few things like "Boltzmann Machine" and tidied it up.
- https://github.com/mohammadpz/pytorch_forward_forward (1.2k stars) -- nice diagrams
- https://github.com/JacksonWuxs/Forward-Forward-Network might be better
- https://github.com/rmwkwok/forward_forward_algorithm <-- last commit was 3 days ago, so this one's hot from the oven. Nice 'pen' colour-diags too. And thorough testing.
^ Great rundown on Twitter with pics!