AI Without Python
An Intro to Machine Learning for C++ Programmers
Background
Machine Learning in 2023
The Current Big Thing™ in software
Whisper, DALL·E, Craiyon 🖍, ChatGPT, GPT-J, LLaMa 🦙, LaMDA, Midjourney, Falcon LLM 🦅, Stable Diffusion, Unstable Diffusion 😉, GitHub Copilot, StarCoder, BERT 🐻, SAM, Chinchilla 🐭
…
A Cambrian explosion of AI tools
... and startups
... and software as a whole
There's something new every day.
(this talk will probably be outdated by tomorrow)This software is no magic
Python Stacks
- The big fish: PyTorch and TensorFlow
- Every ML framwork has a Python front end
- Why Python?
⚠️ Personal opinion time ⚠️
Modern ML software
Objective Truth
Python is slow
"No it's not slow!"
- "This Python program is faster than its C++ equivalent!"
- "
.pycshould do it" - "Python is the most optimized interpreter there is!"
- "Python JIT compilers work!"
- "No matter. The low-level framework does the actual work."
Opaque Frameworks
- Data flow suffers
- Tweaks are hard to impossible
- Many similarities with game engines
- Bloat intensifies
I'm not the only one with such problems
A Crash Course in ML
I am not an ML engineer
So, this is my perspective...
NOPE
Neural Networks
What is a neural network?
It's a function
enum thing { ... };
thing classifier(const image& input);
enum thing { ... };
struct result {
thing t;
float p;
}
std::vector<result> classifier(const image& input);
std::string gpt(const std::string& input);
using gpt_callback = std::function<void(const std::string&)>
void gpt(const std::string& input, gpt_callback cb);
What is a neural network?
It's a computation with parameters
enum thing { ... };
thing classifier(const image& input, const std::vector<float>& parameters);
Parameters
LLaMa-7B - the LLaMa model with 7 billion parameters
NOPE
NOPE
What is a neural network?
A network of neurons, duh
$y = g \left( \sum_{i=1}^{n} w_j x_j + b \right)$
Layers ("deep" means more than 2)
Wait! I know this
$\begin{pmatrix} y_1 \\ y_2 \\ y_3 \end{pmatrix} = g \left( \begin{pmatrix} w_{11} & w_{12} & w_{13} & w_{14} \\ w_{21} & w_{22} & w_{23} & w_{24} \\ w_{31} & w_{32} & w_{33} & w_{14} \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{pmatrix} + \begin{pmatrix} b_1 \\ b_2 \\ b_3 \end{pmatrix} \right)$
Yes. This is mostly everything
Convolution
Neurons don't depend on the entire input
Weights are shared
Feature maps
Pooling
(Subsampling)
Collecting "important" features
What is a neural network?
A collection of layers which define a computation
Terminology time
LeNet
AI like it's 1998
MNIST dataset
LeNet Model
Classify individual hand-wrritten digits
class LeNet(nn.Module):
def __init__(self):
super(LeNet, self).__init__()
self.convs = nn.Sequential(
nn.Conv2d(in_channels=1, out_channels=4, kernel_size=(5, 5)),
nn.Tanh(),
nn.AvgPool2d(2, 2),
nn.Conv2d(in_channels=4, out_channels=12, kernel_size=(5, 5)),
nn.Tanh(),
nn.AvgPool2d(2, 2)
)
self.linear = nn.Sequential(
nn.Linear(4*4*12,10)
)
def forward(self, x: torch.Tensor):
x = self.convs(x)
x = torch.flatten(x, 1)
x = self.linear(x)
return nn.functional.softmax(x, dim = 0)
m_input = create_tensor("input", {28, 28, 1});
ggml_tensor* next;
auto conv0_weight = create_weight_tensor("conv0_weight", {5, 5, 1, 4});
auto conv0_bias = create_weight_tensor("conv0_bias", {1, 1, 4});
next = ggml_conv_2d(m_ctx, conv0_weight, m_input, 1, 1, 0, 0, 1, 1);
next = ggml_add(m_ctx, next, ggml_repeat(m_ctx, conv0_bias, next));
next = ggml_tanh(m_ctx, next);
next = ggml_pool_2d(m_ctx, next, GGML_OP_POOL_AVG, 2, 2, 2, 2, 0, 0);
auto conv1_weight = create_weight_tensor("conv1_weight", {5, 5, 4, 12});
auto conv1_bias = create_weight_tensor("conv1_bias", {1, 1, 12});
next = ggml_conv_2d(m_ctx, conv1_weight, next, 1, 1, 0, 0, 1, 1);
next = ggml_add(m_ctx, next, ggml_repeat(m_ctx, conv1_bias, next));
next = ggml_tanh(m_ctx, next);
next = ggml_pool_2d(m_ctx, next, GGML_OP_POOL_AVG, 2, 2, 2, 2, 0, 0);
next = ggml_reshape_1d(m_ctx, next, 12 * 4 * 4);
auto linear_weight = create_weight_tensor("linear_weight", {12 * 4 * 4, 10});
auto linear_bias = create_weight_tensor("linear_bias", {10});
next = ggml_mul_mat(m_ctx, linear_weight, next);
next = ggml_add(m_ctx, next, linear_bias);
m_output = ggml_soft_max(m_ctx, next);
Practical Challenges With Inference
How to Start?
First, forget about training!
Implement a simple model in the most naive way!
Yes, play with Python, too
You are needed!
Let's ride the hype wave!
End
Questions?
- These slides: ibob.bg/slides/ai-cpp-intro/
- Borislav Stanimirov / ibob.bg / @stanimirovb
Slides license: CC-BY 4.0 