Ibrahim Rayamah — writings

Why Africa is poor

Ibrahim Rayamah — Tue, 18 Aug 2026 00:00:00 GMT

The question

Africa is the poorest continent on earth, and as the ::factories=factory:: of Asia finished developing, it kept falling further behind. Most of the world's extreme poverty now sits in Sub-Saharan Africa, and the income gap between a lot of African countries and rich ones has widened to a factor of 40 or 50. Of course Treating a continent as one economic case study is a silly thing to do. There are 54 countries in Africa with their own governments, currencies and problems, and asking one question about all of them is like assuming the US and Ecuador run on the same machinery. But poverty is the one thing that is close to universal here. There are outliers like Mauritius, the Seychelles and Botswana, and even then the richest country on the continent by GDP per capita would sit in the lower middle of Europe.

The obvious answer is not enough

The symptoms are well known, i mean ... Political instability, corruption, capital that gets destroyed or seized, which makes ::industry=factory:: risky to build, which means fewer good jobs, which means poorer people, which feeds more instability. Add the modern version on top, where open trade makes it harder for young African industries to compete and easier for the best African workers to leave for a better salary somewhere else.

Land is a factor of production, and in the early game it is the important one, because good land feeds more people, and more people means some of them can stop farming and go discover new things. Africa has plenty of land, what it did not have was easy land. The ::desert:: splits the continent east to west, and for most of human history the Sahara worked like an ocean, which left everything south of it cut off from the trade and the ideas moving between Europe, the Middle East and Asia. That single feature is why economists usually split North Africa off from Sub-Saharan Africa and treat them as separate stories.

The soil is even worse, green parts of the centre have fragile ::soil=crop:: that erodes fast unless you manage it with techniques nobody had a thousand years ago. There was not much reason to farm, foraging was simply cheaper and the lush parts came with tropical ::disease::, insects and predators. Anthropologists have a name for the thing that did not happen here, the trap of sedentism. A group finds somewhere with fresh ::water::, wood and no predators, settles in for generations, eats the place clean, and now farming is the only option left because they already forgot how to move. Those gardens of eden barely existed in Africa, so groups stayed small and kept walking.

Fast forward and you get a continent with hundreds of languages and thousands of ethnic groups, drawn over with borders somebody else invented. Many countries ended up landlocked with no ocean access, connected to their neighbours by dirt ::roads=road::, or by ::railways=rail:: left behind by empires that built them to haul ore to a ::port::, not to build an economy. Much of the west coast is raised and awkward to build shipping ::ports=port:: on. So before anyone was even here, this was not a great spawn point.

But geography is not the answer

The economists who dug into this hardest, Daron Acemoglu and James A. Robinson, asked whether geography explains the size of the gap, and their answer was a flat no. Geography matters, you cannot understand Saudi Arabia without ::oil:: or Ukraine without who it is standing next to, but institutions also matter more.

Their favourite example is the ::wheel::. Even after wheels were common knowledge, a lot of groups kept moving heavy things on their heads, which is wildly less efficient than a hand cart. That looks irrational until you look at who was in charge. Rulers taxed by decree and took what they wanted by force, so groups that wanted to be left alone moved away from the few ::roads=road:: that existed, and nobody wanted to build a nice cart that could just be confiscated. The ruling institutions were quietly paying people to stay poor and self sufficient, and that habit outlived them.

Guns and extraction

Atlantic ::ships=ship:: needed labour because the plantations in the Americas were burning through people faster than they could be replaced. Some African rulers already used forced labour and were willing to trade captives for European ::guns::. That trade was self reinforcing in the ugliest way, because a gun is both property and the tool that enforces property, so it was the one piece of technology a strong man could hold onto. Powerful groups got more powerful by selling their rivals, and the money bought more guns.

Europe did not colonise the interior at that point because African armies on home ground with malaria on their side were genuinely expensive to fight. Sailing across an ocean was easier, which tells you something. Then steam ::ships=ship::, industrial ::steel:: and basic malaria treatment changed the maths, and the continent went from trading partner to target. When the empires left, they took the ::tools=tool::, the management and the industrial relations with them, and ::education=school:: was close to nonexistent and worse, they had spent decades hollowing out whatever legitimate local institutions existed and replacing them with a thin administrative shell that reached the capital and nothing else. That shell was useless for building an economy.

The piggy bank problem

The ::diamonds=diamond::, the ::oil::, the metals, the rare earths the whole world now needs for batteries, trillions of dollars sitting in the ground. In a lot of countries that wealth became the ruler's ::money:: for staying in power, not the country's budget for building ::schools=school:: and ::ports=port::, and it compounds into a reputation. Any project in Africa gets priced as riskier than the same project anywhere else, so obviously good infrastructure never gets funded. We saw this with the pipeline stalling in Niger, one of the easiest returns on investment you could draw up, and still not worth it once the politics turned. High global interest rates make it worse, because if you can park cash somewhere safe and still earn a solid return, why take the risk at all.

Botswana

Now let us move to the optimistic part, because that paper is over a decade old. Botswana has every excuse. Landlocked, hard to move around, dirt poor at independence with almost no ::roads=road:: and barely any educated elite, and it has diamonds, which for other countries has been a curse. It became one of the fastest growing economies in the world over 60 years, because it built the boring stuff. Stable property rights, ::courts=court:: that did not just serve one group, a ::democracy=vote:: that mostly worked, real spending on people and ::education=school::, and diamond revenue managed like it belonged to the country.

It is worth remembering that 50 years ago most of Asia looked like much of Africa does today, and 300 years ago, which is nothing, the richest western countries produced about what the continent produces now. Economic success feeds on itself, and so does failure, which means most economies in history were stagnant right up until they were not. If Africa's turn comes, it will look sudden, and it will not have been.

Sources

Acemoglu Robinson 2010 Why is Africa poor Economic History of Developing Regions
Acemoglu Robinson 2012 Why Nations Fail Crown Business
Acemoglu Johnson Robinson 2001 The colonial origins of comparative development American Economic Review
Nunn 2008 The long term effects of Africas slave trades Quarterly Journal of Economics
World Bank 2025 Poverty and Shared Prosperity data

Global path dependence

Ibrahim Rayamah — Tue, 04 Aug 2026 00:00:00 GMT

The question

Why is the world the way it is today, with America and Europe leading, and the majority of the rest of the world behind, how did Europeans conquer the world?

I hear a lot of talk and complaints about colonization, but why didn't Africa or the Americas colonize Europe instead of the other way around? One side had ::steel::, ::swords::, ::guns::, ::horses::, oceangoing ::ships::, ::writing:: systems, and centralized ::kingdoms::, the other side had mostly ::stone:: and wooden weapons. To understand how some societies won the geographic lottery, we need to rewind the clock to about 11,000 BC.

Same starting line

At that point, humans everywhere were remarkably similar, they were Homo sapiens with basically the same brains, the same cognitive abilities, and the same potential. There were minor differences like skin color, body type, or adaptations to local climates, but a baby born in Europe had no advantage over a baby born in Australia, the Americas, or Africa, everyone was a hunter-gatherer. So if everyone started at roughly the same place, how did things diverge so dramatically?

The IQ myth

Most ignorant people say well, Native Americans and Africans had lower IQs and that's why they never caught up with Europe.

But modern IQ tests measure abstract reasoning, literacy, and mathematical concepts taught in schools, they were designed by academics to predict success in industrialized education systems. A researcher spent years working in New Guinea, and one thing that struck him immediately was that people there were exceptionally observant, inventive, and mentally sharp. To survive you had to know which plants were poisonous, how to read the weather, how to track animals, and mistakes would get you killed. If you gave a New Guinean farmer an IQ test designed for an American suburb, he'd fail, obviously, he has never seen the references or types of problems the test assumes. But if you gave a Harvard graduate a New Guinea "IQ test" and made him track an animal through the forest or navigate without a GPS, he'd fail with flying colours too.

Geography determines which societies can develop agriculture, and agriculture determines everything that comes after. The societies that figured it out got a massive head start by thousands of years, and by the time Europeans crossed the Atlantic with guns and steel, they had a 5,000-year advantage.

Why hunter-gatherers couldn't just catch up

Why couldn't hunter-gatherers develop their own complex civilizations?Because they have to stay mobile, they follow herds and track seasonal plants. If food moves, they move, and that mobility creates constraints.

Everyone is in food-gathering mode. Calories are tough to come by, and they take a tribe's time and effort. When you're constantly on the move, you can't accumulate possessions, you can't store food long term, everything you own has to be carried on your back, that means no heavy ::pottery::, ::tools::, or building materials. A nomadic mother can only carry one ::child=baby:: while she's also gathering food and hauling possessions, so hunter-gatherer women spaced births out about four years apart. Settled farmers spaced births about every two years, that's half the population growth rate right there, and over centuries that difference compounds into massive population gaps. If you break your leg in a settled farming village, you heal, but break your leg as a nomad and you're a burden the group can't afford, either they leave you behind, or the whole group slows down and risks starvation. The same goes for the elderly or the sick. And forget about developing complex technology that requires time to tinker, experiment, and fail, it's time you don't have when you're spending every day searching for food. The first societies to stop moving were the first to start building. Agriculture allowed people to settle down, accumulate surplus food, and that surplus changed everything.

The plant jackpot

You might be thinking, okay, so why didn't everyone just figure out agriculture at the same time, how hard can it be to plant some seeds?

As it turns out, it's really hard. Agriculture didn't happen everywhere at once, it happened in a handful of specific places, and those places shared something important, they hit the plant and animal jackpot. Most wild plants are inedible, poisonous, or so low in calories that they're not worth cultivating. Out of hundreds of thousands of plant species on Earth, only a few dozen ever made good crops. The Fertile Crescent had a whole bunch of them. The region stretching from modern-day Egypt through Israel, Lebanon, Syria, and down into Iraq and Iran held wild ::wheat::, ::barley::, ::peas::, and ::lentils:: growing naturally, and those weren't finicky plants, they had large ::seeds::, predictable growth cycles, and they could be replanted easily. China got lucky too with rice and millet, both solid crops that could support dense populations.

The Americas eventually developed ::corn::, but wild corn, called teosinte, was basically useless. It took Native Americans about 3,000 years of selective breeding to turn it into something that resembled modern corn. Think about that, while one group spent thousands of years slowly improving a single crop, people in the Fertile Crescent were already building cities. Sub-Saharan Africa had sorghum, a decent crop, but Africa had fewer plant species to choose from overall. The Sahara also created a massive barrier that slowed the spread of agriculture and information from the Fertile Crescent. Australia essentially got nothing that could be farmed. About 33 of the world's 56 most valuable wild grasses grew in the Fertile Crescent, that's why agriculture started there first, they had the raw materials.

The animal jackpot

Plants are only half the story, animals matter just as much. There are lots of species, but only a few are easy to handle. Throughout human history, only about 14 large mammal species have ever been successfully domesticated. There are strict requirements, it has to grow fast and breed in captivity, and it needs a calm enough disposition so it won't kill you.

Again, Eurasia hit the jackpot with ::sheep::, ::goats::, ::pigs::, ::cows::, ::horses::, ::donkeys::, ::camels::. That's seven of the big 14 right there. South America had llamas and alpacas. North America, Africa, and Australia had zero.

It's not like people didn't try. Native Americans desperately wanted to domesticate bison. Africans tried with zebras and antelope, but it didn't work. Zebras look like stripey ::horses::, but they're vicious, they bite and kick hard enough to kill a lion. Even today, with all our knowledge and tech, we still can't domesticate zebras on a broad scale. Horses are trainable, zebras just aren't wired that way. This wasn't a failure of effort or ingenuity, it was biology and luck.

East west vs north south

Geography shaped one more crucial advantage. Eurasia runs east west. You can travel about 5,000 miles from Portugal to China and stay at roughly the same latitude, same climate, same growing season, same day length. ::Wheat:: domesticated in the Fertile Crescent could spread to Europe, India, and China. ::Farmers:: didn't have to reinvent agriculture, they could take what worked in one place and plant it somewhere else.

Africa and the Americas run north south. Try taking ::corn:: from Mexico and planting it in Canada, it doesn't work, different growing seasons, different climates. Crops hit climate-zone barriers and stop spreading. Africa has the same problem, especially with the Sahara cutting it in half, that means thousands of years of agricultural knowledge that never spread. While Eurasian farmers were swapping seeds and sharing innovations across an entire continent, American and African farmers were often stuck starting from scratch.

Surplus builds empires

Once agriculture takes hold, everything changes, because now there's a food surplus, and that surplus builds empires. When 20 farmers can feed a hundred people, not everyone needs to spend the day hunting and gathering. Surplus gives you extra calories and extra time, it creates specialists. In a hunter-gatherer band, everyone hunts and gathers, there's no room for someone who sits around thinking about better tools. Once you have agriculture, you can afford people who don't produce food. One of the first things you need with surplus food is a way to store it permanently, then you need people to guard it from animals or other tribes, someone has to organize distribution, and the basics of government start to emerge. Later, professional ::armies:: conquer neighbors, and those conquests bring more land, more resources, more ::people=population::, civilization grows.

Competition and connectivity

Here's where political structure becomes crucial. Eurasia was fragmented in a way that actually helped ideas spread. It wasn't one giant unified empire, but competing kingdoms, city-states, and feudal territories connected by trade and migration. When one ::kingdom:: invented something useful like better ::plows::, bronze weapons, or gunpowder, that knowledge spread. Merchants carried innovations along routes, craftsmen moved between territories, and with important technology, you either adapted or someone took you over with it.

The east west axis meant those innovations could travel thousands of miles without hitting climate barriers. Gunpowder invented in China reached Europe, steelmaking techniques developed in the Middle East spread to India and Europe. The Americas didn't have that same connectivity. The Maya developed writing, but it never spread to the Inca or the Aztecs.

China had the opposite problem of too much unity. When China's emperor decided to halt oceanic exploration in the 1400s, that decision stuck for the entire empire. In Europe, when one kingdom stopped exploring, others continued. Spain's rivals couldn't afford to let Spain monopolize the New World. Technology builds on itself. Eurasia moved from stone ::tools:: to copper to bronze to ::iron=steel::. When Cortés arrived in the Americas in the 1500s, the Aztecs were still using stone-age tools. They had obsidian blades, which weren't terrible, but against steel swords it wasn't even close. That wasn't because the Aztecs were dumb or lazy, they started late, they didn't have Eurasia's 5,000-year head start. The Fertile Crescent got agriculture around 8500 BC, Mesoamerica didn't get it until around 3500 BC.

Put that in modern terms. Imagine the United States being attacked by a civilization just a hundred years younger, the 1926 US Army attacking modern America, it wouldn't be a fight. Now reverse it, imagine a society 5,000 years more advanced attacking us, I can't even fathom what that looks like. That's the situation the Incas, Native Americans, and many African societies faced when Europeans arrived, they were fighting enemies millennia ahead of them.

Germs, the deadliest weapon

But the most fearsome thing Europeans brought to the Americas wasn't guns, steel, or horses, it was ::germs::, Europe's invisible and unintentional weapon.

Scholars generally agree that over 90% of the indigenous population of the Americas died from disease after European contact. If you lived in a village of a hundred people, within a generation only a handful would still be alive. Smallpox killed more Native Americans than all European weapons combined.

Some empires collapsed before Europeans even showed up. The Inca emperor Huayna Capac died of smallpox in 1527 without ever seeing a Spaniard. The disease traveled faster than the conquistadors, spreading through trade routes and wiping out populations before Europeans arrived to conquer them.

By the time Pizarro marched into Peru, the Inca Empire was already in chaos. A smallpox epidemic had killed the previous emperor and sparked a civil war over succession.

Why not the other way around?

Why didn't Native American diseases wipe out the Europeans, why was it a one-way street? Europeans brought smallpox, measles, flu, typhus, plague, tuberculosis, and more. The main disease Europeans got from the Americas was likely syphilis. The answer goes back to domesticated animals. ::Sheep::, ::pigs::, ::cows::, and the rest weren't just ::food:: and labor, they were ::disease:: factories. Eurasians lived intimately with their animals for thousands of years, same structures, daily handling, shared air, and diseases leap from animals to humans. Smallpox jumped from ::cattle::, flu morphs from diseases in ::pigs:: and ::ducks::, measles came from ::cattle:: and ::sheep::. Native Americans had fewer domesticated animals and never lived in close quarters with them the same way, so they never developed these diseases.

But animals alone weren't enough, you also needed cities. Diseases like measles, smallpox, and tuberculosis are crowd diseases, they need large, dense populations to keep circulating. Measles, for instance, needs on the order of hundreds of thousands of people. In a band of 50 hunter-gatherers moving through a forest, there aren't enough bodies for measles to survive. Agriculture created that density. Thousands of people packed together were the perfect breeding ground. Over centuries, these diseases killed millions across Europe, Asia, and North Africa. But if you survive them once, you're often immune for life. Over thousands of years, Eurasians developed resistance, the Americas had no protection like that.

When Europeans arrived with these invisible ::plagues::, Native Americans had zero defenses. The geographic lottery didn't just determine who had steel and guns, it determined germs as well, and that turned out to be the deadliest weapon of all.

Closing

I started by asking why Europeans took over the world that doesn't mean it will last forever. I will go deeper into the American and African continents next. Stay tuned, Arigato!

The Career Tree

Ibrahim Rayamah — Wed, 10 Jun 2026 00:00:00 GMT

import CareerTreeBoard from '../../components/CareerTreeBoard.tsx'

Intro

Since the beginning of science, careers have popped up based on demand, earlier days required more research and discovery within the basic sciences. Today there are many layers that sit on top of the basic sciences, and those layers create new paths, to find the perfect path its advised to be honest with yourself, consider your basic strengths, cognitive ability and geographic location. For students and new grads, Its important to see your initial career as a portfolio, and not necessarily what you will be doing in the next 10 years, It is of my personal opinion to have 3 fields in mind, two directly connected to your passion and one that is a safety net.

A career is not linear, we are in times when everything moves fast, and the only constant is change, so be ready to adapt and pivot.

The Career Tree

Read it like a family tree, the root is knowledge, each layer builds on the one above it, and you can follow a branch from a science you like down to the jobs it turns into.

How To Find Your Branch

Ignore titles and look at the content of the work, research on what they do all day, and how much control you have. but before that consider how it connects to your university major.

The degree trap

People act like your degree is supposed to decide your whole life, and I think that screws a lot of people up. That said, if you did not start with some advantages, the right degree can help a lot. If you went to a non-target school or do not already have a network, a good program or a highly ranked school can open doors for you. Nowadays, you can move from physics into software, from biology into data, from mechanical engineering into materials science, or from philosophy into product management, as long as you can connect the dots and do the work.

A degree does not decide what you can do. It mostly shapes your first instinct when you face a problem. lets take a problem: "cooling buildings in hot cities without raising rent". Physics starts by finding the mechanism causing the problem, then reduces it to forces, energy, and cause and effect. Biology & Chemistry starts from the natural properties of the existing material, then changes composition to get better performance at lower cost. Engineering starts from constraints, then designs a system that can work safely, repeatedly, and under failure. Math removes the story and turns the problem into variables, limits, and tradeoffs. Economics studies the incentives around the solution and asks whether people have a reason to adopt it. Social sciences ask what habits or trust issues stop people from adopting a cheaper solution if it already exists.

Your personality

Personality will not pick a career for you, but it tells you what will start annoying you. If you hate uncertainty, try to avoid constant chaos like startups, and if you hate routine, avoid big tech or enterprise roles.

Where you work

Where you work changes the job way more than people admit.

Enterprise companies (e.g., Salesforce, SAP, IBM)

Stable, clear, and a little bit allergic to moving fast, good if you like predictability and structure.
Big Tech (e.g., Apple, Google, Microsoft)

Huge scale, strong systems, smart people, and enough process to make you question a few things, great if you like reliability and prestige.
Frontier labs (e.g., OpenAI, DeepMind, Anthropic)

High stakes, fast research, lots of ambiguity, and not much calm, great if you like deep work and uncertainty, and you don't mind the stress.
Corporate companies (e.g., Procter & Gamble, JPMorgan Chase, Coca-Cola)

Usually the best balance, you can have real ownership of projects, and a normal life, but get ready to smile to on monday mornings.
Small companies (e.g., boutique design studios, regional distributors, specialized law firms)

More ownership, faster feedback, and usually less drama than a startup, you still have lots of responsibilities.
Startups (e.g., YC companies, newly funded ventures)

Everything changes fast and half the plan is fake by next month, great if you want responsibility early, sometimes you get an incredible return on investment.

Born to do

You build this over time, you get good at something, people trust you, you get more control, and the work starts to feel truly yours. Dont wait around for passion to arrive fully formed, spoiler: it doesn't. You have to build it, then see if it holds up when times get hard.

Sources

Oh Guay Kim Harold Lee Heo Shin 2014 fit happens globally meta analytic comparison Personnel Psychology
Lewis Rivkin 1999 development of the ONET Interest Profiler National Center for ONET Development
OKeefe Dweck Walton 2018 implicit theories of interest Psychological Science
Hagger McAnally Star 2026 self determination theory and workplace outcomes meta analysis Stress and Health
World Economic Forum 2025 Future of Jobs Report

Building a tiny GPT in Rust

Ibrahim Rayamah — Sat, 02 May 2026 00:00:00 GMT

If you only call an API, the model is a black box. This project is the opposite: one file of Rust, Candle for tensors, and the ideas from Attention Is All You Need boiled down to a decoder only stack (masked self attention, no encoder). That is the same family as GPT: predict the next character given everything so far.

Everything lives in RustGPT on GitHub. This post sticks to simple language and shows the parts that matter in code.

What you need at the top of the file

Errors use anyhow. Training uses candle_core, candle_nn, and rand for batch sampling and generation.

use anyhow::{bail, Context, Result};
use candle_core::{DType, Device, Tensor, D};
use candle_nn as nn;
use candle_nn::{Module, Optimizer};
use rand::distributions::{Distribution, WeightedIndex};
use rand::{Rng, SeedableRng};
use std::collections::HashMap;
use std::fs;

Knobs in one place

All sizes and training settings sit in Config. vocab_size is filled in after you read the text because it depends on how many unique characters you have.

#[derive(Clone)]
struct Config {
    batch_size: usize,
    block_size: usize,
    max_iters: usize,
    eval_interval: usize,
    eval_iters: usize,
    learning_rate: f64,
    n_embd: usize,
    n_head: usize,
    n_layer: usize,
    dropout: f64,
    seed: u64,
    max_new_tokens: usize,
    vocab_size: usize,
    temperature: f32,
    top_k: usize,
}

impl Default for Config {
    fn default() -> Self {
        Self {
            batch_size: 32,
            block_size: 256,
            max_iters: 8000,
            eval_interval: 1000,
            eval_iters: 100,
            learning_rate: 3e-4,
            n_embd: 192,
            n_head: 6,
            n_layer: 4,
            dropout: 0.1,
            seed: 1337,
            max_new_tokens: 500,
            vocab_size: 0,
            temperature: 0.9,
            top_k: 40,
        }
    }
}

Turn `input.txt` into numbers

We keep it simple: character level. Build the alphabet from the file, sort and dedupe, map each character to a u32. The first 90% of tokens are train, the rest are validation.

struct Dataset {
    train: Vec<u32>,
    val: Vec<u32>,
    itos: Vec<char>,
}

fn build_dataset(path: &str) -> Result<Dataset> {
    let text = fs::read_to_string(path).with_context(|| format!("read {path}"))?;
    let mut vocab: Vec<char> = text.chars().collect();
    vocab.sort_unstable();
    vocab.dedup();

    let stoi: HashMap<char, u32> = vocab
        .iter()
        .enumerate()
        .map(|(i, ch)| (*ch, i as u32))
        .collect();
    let itos = vocab;

    let data = encode(&text, &stoi);
    let split = data.len() * 9 / 10;
    Ok(Dataset {
        train: data[..split].to_vec(),
        val: data[split..].to_vec(),
        itos,
    })
}

fn encode(text: &str, stoi: &HashMap<char, u32>) -> Vec<u32> {
    text.chars().map(|ch| stoi[&ch]).collect()
}

fn decode(tokens: &[u32], itos: &[char]) -> String {
    tokens.iter().map(|&i| itos[i as usize]).collect()
}

Batches: predict the next character

For each random start index we take block_size tokens as x, and the same length shifted by one as y. So at every position the model should guess the next token. That is the whole training target.

#[derive(Clone, Copy)]
enum Split {
    Train,
    Val,
}

fn get_batch(
    split: Split,
    data: &Dataset,
    cfg: &Config,
    device: &Device,
    rng: &mut impl Rng,
) -> Result<(Tensor, Tensor)> {
    let source = match split {
        Split::Train => &data.train,
        Split::Val => &data.val,
    };
    if source.len() <= cfg.block_size + 1 {
        bail!("dataset too small for block_size")
    }

    let max_start = source.len() - cfg.block_size - 1;
    let mut x_buf = Vec::with_capacity(cfg.batch_size * cfg.block_size);
    let mut y_buf = Vec::with_capacity(cfg.batch_size * cfg.block_size);

    for _ in 0..cfg.batch_size {
        let start = rng.gen_range(0..max_start);
        x_buf.extend_from_slice(&source[start..start + cfg.block_size]);
        y_buf.extend_from_slice(&source[start + 1..start + 1 + cfg.block_size]);
    }

    let x = Tensor::from_vec(x_buf, (cfg.batch_size, cfg.block_size), device)?;
    let y = Tensor::from_vec(y_buf, (cfg.batch_size, cfg.block_size), device)?;
    Ok((x, y))
}

Loss is plain cross entropy on the flattened logits and integer labels.

fn compute_loss(logits: &Tensor, targets: &Tensor) -> Result<Tensor> {
    let (b, t, c) = logits.dims3()?;
    let logits = logits.reshape((b * t, c))?;
    let targets = targets.reshape((b * t,))?.to_dtype(DType::U32)?;
    Ok(nn::loss::cross_entropy(&logits, &targets)?)
}

The causal mask (no peeking ahead)

Attention would let every position see the whole sentence. For language modeling we must hide the future. The mask is 1 where i >= j and 0 otherwise, so we keep lower triangular attention.

fn causal_mask(t: usize, device: &Device) -> Result<Tensor> {
    let idx = Tensor::arange(0u32, t as u32, device)?;
    let i = idx.reshape((t, 1))?.broadcast_as((t, t))?;
    let j = idx.reshape((1, t))?.broadcast_as((t, t))?;
    Ok(i.ge(&j)?.to_dtype(DType::U8)?)
}

One attention layer (the heart of the paper)

Project input to Q, K, and V, split heads, scale the product of Q and K transposed, apply the mask and softmax, multiply by V, merge heads, project again. Dropout is on during training.

let k_t = k.transpose(2, 3)?;
let wei = (q.matmul(&k_t)? * self.scale)?;

let mask = causal_mask(t, x.device())?
    .unsqueeze(0)?
    .unsqueeze(0)?
    .broadcast_as((b, self.n_head, t, t))?;
let neg = Tensor::full(-1e4f32, (b, self.n_head, t, t), x.device())?;
let wei = mask.where_cond(&wei, &neg)?;

let wei = nn::ops::softmax(&wei, D::Minus1)?;
let wei = self.attn_dropout.forward(&wei, train)?;

let out = wei.matmul(&v)?;

Block and full GPT forward

Each block is pre norm: layer norm, then attention, add residual. Then layer norm, feedforward (expand by 4x, GELU, project back), add residual. The GELU matches the usual GPT 2 style approximation.

fn forward_t(&self, x: &Tensor, train: bool) -> Result<Tensor> {
    let x = x.broadcast_add(&self.sa.forward_t(&self.ln1.forward(x)?, train)?)?;
    let x = x.broadcast_add(&self.ffwd.forward_t(&self.ln2.forward(&x)?, train)?)?;
    Ok(x)
}

The top level adds token embeddings plus position embeddings, runs n_layer blocks, final norm, then lm_head to logits over the vocabulary.

fn forward_t(&self, idx: &Tensor, train: bool) -> Result<Tensor> {
    let (_b, t) = idx.dims2()?;
    let tok_emb = self.token_embedding.forward(idx)?;
    let pos = Tensor::arange(0u32, t as u32, idx.device())?;
    let pos_emb = self.position_embedding.forward(&pos)?;
    let mut x = tok_emb.broadcast_add(&pos_emb)?;
    for block in &self.blocks {
        x = block.forward_t(&x, train)?;
    }
    let x = self.ln_f.forward(&x)?;
    Ok(self.lm_head.forward(&x)?)
}

Training loop

Pick Metal on Apple Silicon if it works, else CPU. Build the VarMap, AdamW, loop max_iters, print train and val loss on a schedule, backward step each time.

let device = Device::new_metal(0).unwrap_or(Device::Cpu);
if matches!(device, Device::Metal(_)) {
    device.set_seed(cfg.seed)?;
}

let varmap = nn::VarMap::new();
let vb = nn::VarBuilder::from_varmap(&varmap, DType::F32, &device);
let model = GPT::new(&cfg, vb)?;

let mut opt = nn::AdamW::new(
    varmap.all_vars(),
    nn::ParamsAdamW {
        lr: cfg.learning_rate,
        ..Default::default()
    },
)?;

let mut rng = rand::rngs::StdRng::seed_from_u64(cfg.seed);

for iter in 0..cfg.max_iters {
    if iter % cfg.eval_interval == 0 || iter == cfg.max_iters - 1 {
        let (train, val) = estimate_loss(&model, &data, &cfg, &device, &mut rng)?;
        println!("step {iter}: train loss {train:.4}, val loss {val:.4}");
    }

    let (xb, yb) = get_batch(Split::Train, &data, &cfg, &device, &mut rng)?;
    let logits = model.forward_t(&xb, true)?;
    let loss = compute_loss(&logits, &yb)?;
    opt.backward_step(&loss)?;
}

Generating text

We seed from a short slice of training data, then repeatedly take the last logits, divide by temperature, keep only top k candidates, softmax on CPU, sample one id, append, and feed the sliding window back in.

let mut logits = last.to_device(&Device::Cpu)?.to_vec1::<f32>()?;
let temp = cfg.temperature.max(1e-4);
for v in &mut logits {
    *v /= temp;
}
apply_top_k(&mut logits, cfg.top_k);
let probs = softmax_cpu(&logits);
let dist = WeightedIndex::new(&probs)?;
let next_id = dist.sample(rng) as u32;
idx.push(next_id);

Saving weights

Weights are dumped as raw f32 bytes plus a small JSON file listing shapes. Good enough to prove you can serialize what you trained. The loop that fills tensors walks varmap.all_vars(), copies each tensor to CPU, reshapes to a flat f32 vector, and packs bytes (see the repo for the full save_model).

let metadata: Vec<(String, Vec<usize>)> = tensors
    .iter()
    .map(|(name, shape, _)| (name.clone(), shape.clone()))
    .collect();

let metadata_json = serde_json::to_string(&metadata)?;
fs::write(&format!("{}.meta.json", path), metadata_json)?;

let mut all_data = Vec::new();
for (_, _, bytes) in &tensors {
    all_data.extend_from_slice(bytes);
}
fs::write(path, &all_data)?;

The .meta.json file keeps the names and shapes so you could load the blob back if you write a loader.

Closing

If you like learning by typing, this setup is small enough to read in an evening and real enough to watch loss go down. Clone RustGPT, drop TinyShakespeare or any text into input.txt, and run cargo run --release. The architecture is the decoder side of the transformer from Attention Is All You Need. The rest is training and sampling.

Early History of AI

Ibrahim Rayamah — Sun, 01 Feb 2026 00:00:00 GMT

Intro

Computers are now doing some of the things that humans do when we say "we are thinking". I am convinced that machines will think in our lifetime and they already are .. sort of "thinking". Below the algorithmic level, I know there is something we are overlooking as humans, something we are denying, "but they are just computers, they can never think like us". Maybe computers will never think like us, bu instead we will outsource our thinking to the extent that there will be no distinction between man and machine in terms of output.

The history of thinking machines dates back to the year 1939, Nazi Germany deploys the Enigma, an encryption machine believed to be unbreakable. The Allied forces face a new challenge and turn to Alan Turing, a visionary mathematician who then envisioned a new kind of machine, which kickstarted the era of computing.

1947

By 1946, every military in the world understands the power of computers and wants one of their own. The problem is efficiency, computers rely on vacuum tubes, which work like giant light bulbs. They require constant maintenance and manual labor, and some military computers grow to the size of entire warehouses. So scientists begin searching for a better way.

A brilliant physicist enters the picture. His name is William Shockley. He imagines powering computers using the element germanium to create semiconductors. A semiconductor sits between conductors like metal and insulators like rubber. It can do both, allowing it to act as an electrical switch [0, 1]. This invention becomes known as the transistor and changes technology forever. Though its applications are still theoretical, the smartest minds in the world immediately see the future. A transistor, no bigger than a kernel of corn, sparks visions of super computers.

1950

In 1950, Alan Turing introduces the Imitation Game, later called the Turing Test. It is designed to determine whether a machine can exhibit intelligent behavior indistinguishable from a human. The test fundamentally questions the limits of artificial and human intelligence.

Universities such as Harvard, MIT, and Princeton begin offering computer science degrees. Computer engineers become some of the highest paid skilled workers in the world and the smartest people flock to these programs.

One of them is Marvin Minsky, a 23 year old PhD candidate at Princeton University. Inspired by Turing’s work, Minsky builds the world’s first working neural network using wires and six vacuum tubes as synapses. Despite frequent failures, the machine works and lays the foundation for artificial intelligence and neural networks. Shockley’s transistor leads to the transistor radio, which becomes the highest selling consumer electronic device of all time. The world gets its first taste of compact electronics. Shockley leaves Bell Labs and starts Shockley Semiconductor in Palo Alto, making it one of the earliest tech startups in Silicon Valley. He recruits bright minds such as Gordon Moore and Robert Noyce, along with twelve other geniuses.

1956

Just months after Shockley received the Nobel Prize, eight of the men he recruited left and founded Fairchild Semiconductor. Fairchild quickly became profitable, selling silicon-based transistors to IBM for military navigation systems. This proved that silicon was far superior to germanium, with profit margins nearly 30 times higher.

Manufacturing Crisis

The military now needs thousands of transistors, but manufacturing is unreliable. A simple knock could destroy a transistor. Jean Hoerni, a physicist at Fairchild, invents the planar process, adding a protective layer to transistors. This dramatically increases durability and scalability. Fairchild holds the patent, forcing anyone making transistors to pay them, and so their profits soar even higher.

Integrated Circuits

Despite success, transistors are limited because each one performs a single task and scaling becomes impossible. Robert Noyce creates the integrated circuit, allowing transistors to operate together on a single chip. Though Jack Kilby proposed a similar idea earlier, Noyce’s version proves practical. This invention creates an entirely new industry. Former Fairchild employees leave to form new companies, later called the “Fairchildren,” including Intel, AMD, and LSI Logic. Over 90 percent of today’s computing power can be traced back to Fairchild.

conclusion

To be continued ..

Where are we going

Ibrahim Rayamah — Sun, 28 Dec 2025 00:00:00 GMT

Introduction

Tim Cook: "I'm not worried about computers thinking like humans, I am more concerned about humans thinking like computers"

The people of today are goal-driven; they move without personal values or compassion. The average 18-25 year old is hopeless, lacks purpose, has no direction. We outsource our thinking and wait for the next influencer, the next reel, the next TikTok to tell us what to do, to give us opinions, because we don't think for ourselves. We are scared to make mistakes, we are scared to go wrong, and even more scared to be criticized.

The Cause

Online perfectionism

Every single video or picture online is of a perfect life, a perfect marriage, perfect scores on exams, perfect makeup. People spend hours perfecting that one post for social media, but when you see it you compare it to your normal life. Don't.

Social media was originally created to be a personal space you can share together with friends even when they're far from you. But it is now a perfection competition: who has the best pictures, the best life, lives in a better country. We make each other feel like we lack something. It makes people insecure to post an imperfect picture, an imperfect life, an imperfect result; but all some people will ever have is an average life, what will become of these people? The guy who has a rusted car will never post it because he fears his imperfection, the girl who can't afford makeup or a good camera will fear to show herself because she's imperfect according to modern standards. They will forever hate their lives because they will never be enough.

Who is to blame? We must blame ourselves.

Individualism

We were promised a better life if we shifted from communities to individualism. Everything became personalized: personalized video recommendations, personalized job recommendations, even people recommendations. But now that every person has the power to become/do whatever they want... we suddenly cannot choose. An endless number of choices: millions of clothes to buy online but you don't know what to buy, millions of men and women you can see on social platforms but you cannot choose who to be with, hundreds of apps, thousands of websites on your phone, but you are bored. Millions of job openings but you're unemployed, millions of careers but you don't know what to be... The human brain did not evolve to evaluate millions of options. It evolved to choose between a few concrete paths inside a tribe.

Fear and hatred

Go to any social media, there are groups of men who are dedicated to blaming women, and groups of women who swear that men are trash. There are groups who are dedicated to hating on other races, other beliefs. Hatred is now a way people can feel part of communities; it is now a brand. When human beings feel inferior they cling to a collective hate campaign, and they blame the opposite group for all their problems: blame men, blame women, blame the government, blame the older generation, blame immigrants. They never seem to look at the root of the problem (themselves). They are scared of being at fault so they always blame others.

Remarks

Stop treating purpose as something you find

Purpose is not discovered by scrolling, thinking, or waiting to feel ready. That mindset is part of the damage.

Pick one direction that is “good enough,” not perfect.
Stay long enough to suffer, learn, and matter.
Accept that boredom and doubt are not signs of failure; they are entry fees.

Chosen constraints

Freedom without limits creates anxiety.

Give yourself fewer apps.
Give yourself fewer platforms.
Give yourself fewer “backup plans.”
Give yourself fewer options.

Be comfortable with your choices; it's okay to be wrong.

Conclusion

We are not lost because the world is broken. We are lost because we refuse limits.

We want freedom without commitment, choice without consequence, identity without responsibility. We want meaning, but we avoid the very things that create it: discipline, sacrifice, and long-term effort. So we scroll, compare, blame, and wait for clarity that never arrives.

The solution is not going backward, and it is not destroying technology. It is learning how to live as humans again inside a world that moves too fast. That means choosing fewer things and taking them seriously. It means building small communities instead of performing for large audiences. It means accepting that an ordinary life lived with responsibility is not a failure. So where are we going? Nowhere, until we act like humans.

Rotation Matrices in Quadcopter dynamics

Ibrahim Rayamah — Thu, 03 Oct 2024 00:00:00 GMT

Introduction

In robotics and aerospace engineering, rotation matrices are vital tools for translating vector coordinates from one frame of reference to another. When applied to systems like quadcopters, they enable easy transformation of vectors from the quadcopter’s body frame to an external, fixed inertial frame[^1].

Definition

A rotation matrix is a mathematical construct used to rotate vectors in three-dimensional space. By applying a rotation matrix, we can convert the representation of a vector from one coordinate system to another.

For instance, a vector expressed in a quadcopter's body frame can be translated to the inertial frame of the ground.

To better understand its significance, consider a quadcopter in flight :

Body Frame: This coordinate system moves and rotates with the quadcopter, with its own x, y, and z axes aligned to the craft’s orientation.
Inertial Frame: This is a fixed coordinate system, usually defined relative to the ground, representing Earth’s x, y, and z axes.

Movement

When the quadcopter performs movements such as yaw, pitch, or roll, vectors in its body frame must be converted to the inertial frame for accurate computation of movement, orientation, and external interactions. This transformation is essential for navigation systems, trajectory planning, and understanding how the quadcopter behaves relative to the Earth.

Constructing the Rotation Matrix:

The rotation matrix 𝑅 is built using Euler angles that represent three successive rotations:

Yaw ( 𝜓 ): Rotation around the z-axis, adjusting the direction in the horizontal plane (e.g., turning left or right).
Pitch ( 𝜃 ): Rotation around the new y-axis, tilting the quadcopter's nose up or down.
Roll ( 𝜙): Rotation around the new x-axis, causing side-to-side tilting.

These rotations give us the full rotation matrix 𝑅:

Thus:

For a given vector x in the body frame, the corresponding vector is given by Rx in the inertial frame

Example Scenario Suppose the quadcopter tilts and rotates mid-flight. You need to know where "forward" points now from the ground's perspective (in the inertial frame). By applying 𝑅 to a vector pointing forward in the body frame, you get a new vector showing where that direction is pointing from the ground's view.

This is crucial for controlling the quadcopter’s motion relative to the ground and understanding how it interacts with the environment.

Conclusion

The quadcopter's motion involves complex kinematics where the body’s position and orientation need to be managed using the speeds of its four rotors. The relationship between body movements and world movements is defined through a rotation matrix 𝑅, which captures how the body frame's orientation maps to the fixed inertial frame. This matrix is crucial for designing controllers that can direct the quadcopter's movement and stabilize it in flight.

[^1]: Rotation matrices transform vectors between a quadcopter's body and external frame.

Embedded Systems & Pulse Oximetry

Ibrahim Rayamah — Tue, 20 Aug 2024 00:00:00 GMT

Introduction

Pulse oximetry is an efficient way to monitor oxygen saturation (SpO2) in the blood. The role of embedded technology here is to process signals in real time to provide accurate SpO2 readings. The pulse oximeter uses principles of light absorption to determine oxygen levels, and the system is responsible for managing sensors, processing data, and calculating the oxygen saturation.

Components

Light Source (LEDs):

These LEDs shine light through the tissue, and the amount of light absorbed depends on the oxygenation levels of the hemoglobin in the blood.

Photodetector:

A photodetector measures the amount of light that is absorbed by the blood. The amount of light absorbed by the blood is inversely proportional to the amount of oxygenated hemoglobin. The photodetector sends this data to the microcontroller.

Microcontroller:

The microcontroller processes the signals from the photodetector to calculate the absorption ratios of red and infrared light. This ratio is crucial in determining the SpO2 level.

The core of the microcontroller's task is to apply Beer-Lambert's Law and convert the measured absorption into a meaningful oxygen saturation value. The absorption of red and infrared light is calculated and compared as:

This ratio is processed by the microcontroller, which then calculates the SpO2 and displays it to the user.

Application

Let’s assume the measured absorbance at red light (660 nm) is A red = 0.4 & at infrared light (940 nm) A infrared = 0.2 The system will calculate it as :

This result means 50% of the hemoglobin in the blood is oxygenated

Oxygen saturation of 50% is significantly lower than the normal range (95-100%). This suggests that the body is not receiving enough oxygen, which could indicate a serious condition such as respiratory failure or severe hypoxemia.

Error Sources

These are some factors can lead to errors in the system:

Carboxyhemoglobin/methemoglobin absorb light similarly to oxyhemoglobin, leading to overestimation of oxygen saturation. The embedded system needs to be calibrated to account for this.
Motion can cause signal distortion. The embedded system filters these out using motion-detection algorithms, ensuring accurate readings even during patient movement.

citation:

*Upadhyay, A., & Dhapola, A. S. (2015). Embedded systems and its application in medical field. [https://doi.org/10.13140/2.1.1299.1528] *

Ibrahim Rayamah — writings

Why Africa is poor

The question

The obvious answer is not enough

But geography is not the answer

Guns and extraction

The piggy bank problem

Botswana

Sources

Global path dependence

The question

Same starting line

The IQ myth

Why hunter-gatherers couldn't just catch up

The plant jackpot

The animal jackpot

East west vs north south

Surplus builds empires

Competition and connectivity

Germs, the deadliest weapon

Why not the other way around?

Closing

The Career Tree

Intro

The Career Tree

How To Find Your Branch

The degree trap

Where you work

Born to do

Sources

Building a tiny GPT in Rust

What you need at the top of the file

Knobs in one place

Turn input.txt into numbers

Batches: predict the next character

The causal mask (no peeking ahead)

One attention layer (the heart of the paper)

Block and full GPT forward

Training loop

Generating text

Saving weights

Closing

Early History of AI

Intro

1947

1950

1956

Manufacturing Crisis

Integrated Circuits

conclusion

Where are we going

Introduction

The Cause

Online perfectionism

Individualism

Fear and hatred

Remarks

Stop treating purpose as something you find

Chosen constraints

Conclusion

Rotation Matrices in Quadcopter dynamics

Introduction

Definition

Movement

Conclusion

Embedded Systems & Pulse Oximetry

Introduction

Components

Application

Error Sources

Turn `input.txt` into numbers