Your attention span is being stripped for parts and sold for scrap metal. From content whiplash before we’ve gotten out of bed to feeling like we’re in a daze when it’s bedtime. The glass screen in our pocket has put us in a self-induced psychosis. I’ve spent the summer unraveling this mess.

Newsfeeds and content feeds are designed for one thing: warping your sense of time. When I take the train to work in the morning I notice how almost everyone is flicking up on Instagram Reels or TikTok. This is because the act of doing something mundane like traveling between point A to point B on public transport is dampened by these apps. It’s time dilation in play.

We never need to know peace or boredom. This is a betrayal to ourselves. The quiet gaps in our day which might’ve resulted in an original thought are now filled with the consumption of mental junk food. Charles Darwin would work for two 90 min periods and then go on two separate 60 min walks. No AirPods, no productivity podcasts, no Newsfeeds. Just his own thoughts and a groundedness in the present moment.

I don’t blame us. It’s a bit like handing a cigarette to a 15 year old and never mentioning the downsides. The biggest problem is that the content on these apps is endless. The days of early Facebook where our timelines were filled with people we know is long behind us.

Netflix’s biggest competition is sleep. - Netflix CEO Reed Hastings.

This isn’t where I recap my summer of living with a dumb phone and deleting all social media profiles. Becoming a Luddite is unrealistic. I have no affinity for a cold Cabin life in Lincoln, Montana. Instead this summer I focused on narrowing the amount of available content. I used ad-block extensions to “delete” parts of the web apps that felt endless. My YouTube homepage is a black page, the subscriptions and YouTube shorts tabs are all deleted. If I want to watch a video I need to seek it out by searching for it rather than being nudged by the Algorithm Overlords.

By killing my infinite feeds, I quickly run out of content to consume. I’m forced to fill the gaps of time in other ways. There are 8 or so creators I truly care for and I have post notifications on for them. This feels like the old internet when RSS readers would push you content you actually wanted.

By spending less time in the digital realm, I feel more present in the physical one. I’m discovering the dwindling art of talking to strangers, reading more books, and spending more time slowly shuffling through unorganized bins of vinyl records. I’ve made a trade. I’m less online but have more vibrant days.

How much of the furniture in your home just randomly appeared one day? Probably none. Almost all of it was curated by you over time. We wouldn’t clutter our homes with random furniture so why are we allowing BigCrops to clutter our minds?

Introducing: Coffee Engineering ☕️

I’ll keep it short because I know your time is valuable. It’s called Coffee Engineering and it lives on: coffee.engineering. The tagline is “caffeine for AI” (fun right?). And the SEO optimized description is “coffee engineering is a tool to create, share, and manage your AI prompts and agents in a simple, easy-to-use no-code way”.

Few nights ago I was having a conversation with my brother. He mentioned sometimes you just have an itch you need to scratch. A landscape in your mind that needs to put onto canvas. A conversation between two friends that needs to be turned into a short story. A little product idea that needs to not die on the cutting room floor. Coffee Engineering is an itch I just had to scratch, and I’m glad I did.

📀 If you’re a visual learner and not a reader, I have a neat demo on YouTube.

Watch Demo Video

Observation: prompts are repetitive

I often go to ChatGPT or Claude and say:

1. You’re a senior React software engineer using Tanstack Router and Material UI v6, give me code that does X.

2. Give me a list of 5 movies similar to movie Y, include director and year in the response.

3. Research X and give me a table that includes name, app, cost.

Prompts tend to be repetitive.

Observation: people want to share their prompts

I know there are entire Discord, Reddit, and Facebook communities for people trying to squeeze the most out of large language models. A member (who might not be too tech savvy) will ask “what is the best way to analyze this CSV” and some other member will reply with a Pastebin or Github gist link saying “this is the prompt I use.”

There are also technical communities like the Cursor Directory where people are sharing prompt files you’re expected to copy-and-paste into your code editor.

In all these communities the expectation is that people float around these text files like loose pieces of paper and copy-and-paste them into AI services. Seems a bit inefficient no?

Observation: more copying-and-pasting

I know Claude is good at code. I know ChatGPT is good at more high level thinking. Sometimes I’ll ask ChatGPT something and then copy and paste that response into Claude and ask it for more details. Hmm… if only there was a more cleaver way to switch in and out of different models while preserving the context.

Observation: Projects and Custom GPTs

Some of what I’ve built can be achieved with Claude Projects or Custom GPTs. When I started working on Coffee Engineering these products didn’t exist. I do think I’ve pushed them further in some ways and made them much more accessible.

Observation: why is there only one prompt?

Every “Custom GPT” style product lets you add some context (from files or in the system prompt) and then the user interacts with it in the same manner: using a single text box.

I wanted to rethink this. Why can’t I have 3 inputs? One is the usual prompt, another is a toggle that chooses whether the AI response should include Emoji’s, and another to choose which language the response should be in. This is what I’m calling caffeinated prompts.

Beans

Beans are little self contained things (I refuse to call them “agents”) that have context built in. (This is where the visual demo might help).

1. You build a Bean by giving a short description of what they do. Example: “A Bean that returns a list of 5 movies similar to the one provided”

2. We take the user description and turn it into a system prompt using best practices and providing better output shape when appropriate

3. You save it as a public bean called “Movie Recommender” and it becomes accessible on “coffee.engineering/@shahzeb/movies”

4. Open the Bean, type in Interstellar hit go and boom everything works.

5. Share the Bean’s link with your mom and now you’re both watching the Martian

I have a Bean to give me Svelete 5 code (which is a frontend framework Coffee is built using). I first ask it for a high level project overview using GPT-4 and then in the same thread I switch to Claude and keep the conversation going. If I don’t like the output a model gave me, with one click I can regenerate the output either using a different model or different temperature (how creative or accurate).

I can search public Beans. My dad made a fun public Bean and I can just search and use it. Search results are ranked by popularity. Naturally, the most subscribed to Beans bubble to the top. This means that Beans marketplace (aka search) self select for better Beans.

I’m learning Spanish right now (shout out to an almost 350 day Duolingo streak). I have a Spanish translation bean. I can right click on a different Bean’s response and select “Send Response” and select the Spanish Bean. So one Bean just sends its output to another. So easy. No copy-and-pasting.

You can also invoke multiple different Beans from your dashboard at the same time. I’ve not seen anyone else do this. That’s why the Coffee Engineering homepage doesn’t look like the other AI company’s.

I’ve poured a lot of love into the experience of Coffee Engineering. Right click on a movie name from the above Movies Bean and you can search Letterboxd or Wikipedia straight from the right-click menu. The product was not vibe coded. That’s why it has been in the works for a while. I’m confident I’m doing things that even the big boys with trillion-dollar evaluations are not.

It is currently free. Sign up and I’ll add you to the beta user list:

Go to Coffee Engineering

It feels really good to earn the right to finish. Not many creative side projects make it this far. The foundation is set and I know there’s a ton I can build on top of this. But now I want to earn the right to build those things.

My next goal is to get 100 users.

Writing is accountability

Even if it’s without an audience. Let me explain.

There was very little progress on my project in January so I ended up skipping the post. Things were more productive in Feb hence this blogpost. Also, I’ve been reflecting on the act of writing.

I think writing is the only thing I find harder than pure software development. In trying to gather my personal notes into something that’ll be fruitful for an audience, I find my neurons lighting up like little LED light bulbs. Now truth be told, considering I have an audience of a few subscribers, I mostly write these monthly reflections for myself.

Knowing that at the end of the month I’ll need to share things I learned forces me to keep momentum. That was my lesson from January. Since there wasn’t much to share, there was no devlog post. But it didn’t feel nice break my little monthly streak. Lesson learned. Always make progress and share it regardless of scope. Onwards.

Product development is frontend development

Admittedly, things are behind where I had anticipated, and I think I have a diagnosis.

Backend engineering is much easier than frontend engineering. I think there are 2 reasons for this:

1. You can always make the frontend better. There’s always some little corner of your product that you can perfect down to the pixel. Backends (especially those that are pre-customer and hence pre-scale) don’t have this problem. After an API endpoint or some backend infrastructure is functional, it’s effectively done. The frontend is never done.

2. Frontend engineering is mostly subjective, not objective. This is closely related to the previous point. When you create an API endpoint, you can unit test it for correctness. With a new UI that’s truly doing something innovative, how do you unit test it for correctness? I don’t mean a snapshot test or a test to make sure the correct elements are rendered to the screen, I mean, that the users will know it’s correct.

If you’ve experienced the above listed symptoms then you might be in a kind of product development hell. It’s a hell of our own creation, and there might be a solution.

Leave things on the cutting floor

The remedy is simple: accept that things might not be perfect. Realize that doing refactors when there are 0 users is a waste of time. Realize that trying to design component in a perfect reusable fashion only to never re-use them was wasted effort.

I propose a kind of Occam's razor of product development: if I didn’t change or fix it, would anyone complain? Often the answer is no.

The above frame of thinking has helped me push through some challenging UI issues on my solo development journey. Not only does this help in prioritization but I’ve decided to cut entire features out of the first version of my product. Getting out there is more important than never getting out there but having a few more half-baked features.

Hoping to release the beta version of my product end-of-month. Keen to share it.

The back and forth

Last month was all about getting a functional MVP. This required me to switch back-and-froth between the backend and the frontend to make sure everything was working. This month required a change of strategy.

There are two ways of thinking about progress. If you’re an artist trying to improve your craft, you might choose to work on the skill of painting and the skill of ceramics in a given month. Or you can fully commit to just improving ceramics. Both strategies are going to result in progress but there’s a different cost associated with each strategy.

Last month I got the MVP running, this month I decided to change strategy on making progress. The back-and-forth between implementing a backend endpoint and then switching gears to the frontend UI task just didn’t feel right. The context switching wasn’t resulting in the type of progress I wanted.

I completed all my forecasted goals and I think choosing to just work on one craft (backend) this month paid off.

It’s a Sisyphean task

The act of building something is the act of getting to know oneself. The more progress you make, the more blind spots you reveal. It’s easy to get overwhelmed by the seemingly endless todo tasks. Here’s one strategy I started at the start of this month that will help.

Make a document and write down “Version 1” at the top. Add a table bellow with the following columns: a short description of the feature, a “page” column that links to more details about that feature, and a “status” column that tracks the progress or completion. I use Obsidian, here’s an example if I was building a blogging platform:

Each one of those high-level features get their own document. You don’t need to conform to any template for these documents, you just jot down what it would take for that feature to be complete. If we click on `feature- draft posts` we might see this document:

Now, not only do you know when your product Version 1 will be ready for release, but you can break down the individual features into items that will be easy to conquer. Feature creep can also be prevented because you have a roadmap for this particular release.

Maybe now, one must imagine Sisyphus happy.

The push and pull of simplicity

I’ve been dabbling with Cursor. A code editor that has “AI code generation” built into it. It’s really impressive when it comes to boilerplate code generation. I won’t lie it is impressive but I think the hardest parts of product development still require the creator to do some deep thinking and problem solving.

Here are some shortcomings of AI code generation tools:

1. If you’re unfamiliar with your own code or tools, then AI will trick you. You need familiarity with your tools to build the intuition to pick up on possibly incorrect AI output.

2. Personal skill atrophy: probably the biggest risk of these tools. You’ll forget the skill of programming and become a “prompt engineer”. Maybe this is the future? I’m not sure. I need to reflect a bit more on this.

3. Complexity will slip in and you won’t even realize it. I asked Cursor to generate some boilerplate Stripe Billing checkout code and it created code with 3 different pricing tiers. Billing is complex and if I choose to blindly accept 3 different pricing tiers because my codegen tool outputted it, then I take on serious maintenance and technical debt risk.

Often the most elegant solutions are ones you need to think about. Solutions that can’t be generated regardless of large context windows.

Pricing a product is an art not a science

This is the first thing I’ll ship that’ll have at least one priced tier. This month was the first time I created a Stripe account and started to read up pricing strategies for digital products. It turns out that pricing isn’t an exact science.

One of the first things I did was ask Claude and GPT to assist with a pricing strategy. They’re kind of bad at this task. In part because they choose complexity (see previous section) or maybe they’re just bad at math.

I learnt that nothing beats a simple Excel file. Track the parts that’ll cost and then add a margin on top of it. Stripe has a lot of bells and whistles (do you want users to have free trials? coupons? monthly vs. annual discounts?). My recommendation is to start things simple. The more layers of billing code you add, the more time you’ll spend moonlighting as a “billing engineer”.

I did gain one important insight in my readings: price things higher than you’re comfortable with. Reducing the price is easier than increasing it.

Boredom is a feature, not a bug

December is a sleepy month. The perfect month to watch Japanese film Perfect Days that highlights a character in Tokyo finding the beauty in a simple life. I think we can all learn from this film.

In order to make progress all you need is attention. But at the same time, some of the best insights will not come when you’re sitting in front of your code-editor. Going on walks. Writing (which is perhaps the only thing I find harder than coding). And just having other hobbies is a crucial part of making progress.

Happy New Year. Talk to you in a month my friend.

Your domain is about to expire

A month ago I received an email from Vercel telling me that my domain was about to expire. Getting a domain expiration email always feels embarrassing. It's an unexpected reminder of what could've been. You recall a year ago when you ran `npx create-react-app world-changing-idea` only to forget about it 3 days later.

The crossroads: let the domain disappear into the ether like it never happened, or renew? Let's renew.

The proof is in the weekend

One of my favorite essays is "How to Build Anything Extremely Quickly". The gist is simple: outline the tasks and speed run them. A complicated task? Break it up into smaller tasks. Speed run those.

I used this method to build a weekend prototype of the core functionality of the thing. No SAAS landing page, no login, or database. Just a simple frontend with one component that hits a single Hono endpoint. It just works. This proved to me that thing could in fact be a functional product.

Don't obsess about your knives

There's a tendency in software engineering to obsess about our tools. Our stacks. Our frameworks. Our Neovim or Obsidian configs. I think tooling matters, but time spent perfecting your tooling is time wasted. I'm saying this having spent hours on YouTube trying to nail the "perfect" Neovim setup or Obsidian setup .

When tools have infinite customizability, our brain tricks us into thinking: surely if I configure this tool "the perfect way" I'll have superpowers. Our pursuit of perfect tooling will only lead to a dead-end. I doubt chefs spend this much time obsessing about their knives when they have a dish to make.

The zen of avoiding the cool thing

After completing my weekend proof of concept, I wanted to start building the actual product. I thought about using NextJS for this application but I think I have Javascript fatigue. I know building a thing using Next is probably what the cool kids would do but in the words of one of my favorite artists:

You can't be too cool, because then you'll just freeze to death.

- Tyler the Creator.

I'm choosing not to do the cool thing. I'm building the application using Laravel (which is like Ruby on Rails) but for PHP. It gives me authentication, queues, workers, and sockets all out of the box.

In the JS ecosystem simple (yet core) parts of the application end up requiring entire companies. Like Clerk for authentication, Pusher for real time sockets, and Trigger for queues and workers.

Maybe I'm jaded on the JS ecosystem but I know that each `npm install package` will inevitably come with issues. Either if it's needing to eject from the predefined happy path of these third party services, or their lack of documentation, or their cost, or their downtime.

Find your dependencies and eliminate them.

- Excel team moto by Joel Spolsky.

Nginx on a Saturday morning

Taking daily notes on development can paint a picture of overcoming difficult things. You inevitably come across a technical problem that feels like work.

For me, a couple Saturday mornings ago, I was trying to get an Nginx config wired the right way to work with SSL and CloudFare tunnels. What the words in the previous sentence mean doesn't really matter but what matters is that it was tough. But you put the work in, try different methods, and come out to the other side with some new scar tissue. The breakage of tissue is how muscle grows I'm told.

Earning the right to work on things

There's always a little rabbit telling Alice to go down a different hole. I'm Alice. The never-ending technical SAAS products is my rabbit hole.

One Tuesday evening I spent an hour looking at the perfect no-code Business Intelligence tool. Something that could give me insight into a product's user behavior. I have zero users. I barely have a product. Why was I wasting my time? I need to earn the right to work on certain things.

It's all distraction. Avoid it.

Every 3 months at the company All-Hands an intern asks the CTO if we can “rewrite our entire codebase in Rust?” And every time I wonder, what the heck is Rust.

I wanted to relate to my fellow interns so I decided to learn Rust. The following are thoughts from someone who writes way too much TypeScript and Python attempting to learn Rust over the past ~2 months. Strap in, it’s gonna be a real gathering of the minds.

If you just want to see what I built, and the code for it:

View the app

View the code

Foundations.

Rust makes getting started extremely easy. Run the terminal command on rustup and the Rust toolchain will be installed. Rust is a fairly mature language with a rich amount of high quality open-source educational material. I learnt most of the fundamentals using the Rust Programming Language book and Rust By Examples (both free online resources).

If you’re getting started, I recommend reading the first 4 chapters of the Programming Language book and then trying to build something yourself. When you come across an error or new concept, CTRL+F between the ebook book and the examples website.

Why build a “Redis clone”?

The only way to learn a language is to try and build something using it. Advanced concepts such as the “Rust borrow checker” aren’t going to make sense just by reading over some example code.

To me, Redis is kind of the perfect tool to try and recreate. In the real world it’s used as a caching system / data store / queuing system. But at its core: you give Redis a string → it either gets or changes some data → it returns some data. Easy.

This core functionality of Redis is simple enough that it can be turned into a straightforward coding interview question. You can have a basic solution in < 30 lines of JS. A bit of string parsing, mutate some objects, and return a string or number. Simple. But what if you want to add on complexity. How would you create a system that neatly handles 40+ commands? How would you handle creating a server that can execute these commands over HTTP? What about data persistence and importing / exporting data? Replicating data across nodes? What about making the same code run both on the CLI and on the web (spoiler alert)?

That’s why I kind of love Redis. It’s so simple yet you can choose to learn new things just by adding complexity.

So I decided to build a Redis CLI tool with the goal of learning:

1. Fundamentals of Rust

2. How it handles basic user input and output

3. How to manipulate and pass around data

4. How Rust handles inheritance for different commands

5. Finally, can we get Rust to run on the web

Here’s a list of things I learnt while working on my project, roughly in the order that I came across them.

A compiler that tells you what’s wrong, and how to fix it.

Like C++, Rust is a complied language. Which means there is a compiler involved. (Gathering of the minds, I told you). I’ve done a lot of C++ in my University days and I don’t miss seg faults, weird memory pointer issues (turns out it was actually a skill issue), super verbose typing, and using gdb at 1am. How much of that spills over into the Rust world?

Well, let’s start with the compiler. When you run `cargo run` it’ll build and run your code. This same command is when any build time issues will popup. And popup they will.

We’ll talk later about what this error actually means, but notice the fact that there is a “help” section where it tells you what you need to add (in the green text). This is a common trend with learning Rust: write a thing → compiler error → learn new concept → implement fix based on compiler error.

It’s typed sensibly.

The neat thing about Rust is that the language does a great job inferring the types. In fact, it does it much better than TypeScript.

Consider this TypeScript code (TS playground link):

Notice how we have a function returning a DataType. We have a map called `data` we return but `data` still has “any” as both the inferred key and value. It’s unable to infer that the `data` should be `Map<string, number>` based on the return type of the function.

Now, let’s see Rust in action (Git codelink):

Same exact function here but written in Rust. I didn’t actually add typing to `data` but it was correctly inferred `HashMap<String, i64>` based on the return type of the function. (The only reason you see it in gray is because the VSCode Rust plugin shows inferred types).

Amazing. This makes so much sense. This was a consistent theme while learning Rust: it made me realize TypeScript is the Temu typing system. Actually, that’s probably Python. We don’t need to talk about Python typing.

Protect you from you.

A major selling point of Rust is safety when it comes to memory access. C++ lets you pass around pointers and do whatever you want with them. Want to cast a Dog pointer into a Chair pointer? Cool, go for it. Turn a Train class object into a List of Dolphins? Sure. In C++ when you handoff a pointer in a function call, anything is possible. I think this is partially where the “let’s re-write it in Rust” meme comes from. Rust is much more strict about the way data is passed around and reassigned. It’s safer.

Part of what enables this memory safety is the Rust borrow checker. This is one system that can guarantee safety, but it comes at a cost: the learning curve.

Let’s see it in action. Let’s create a function called `execute` that pints out key and value.

fn execute(&self, _: &mut DataType) {
  let key = self.key;
  println!("{}", key)
}

First we add key. Then we copy and paste the key line and change the “key” to value” for something like this:

Wait, what’s this? We’re getting an error. I have an intentional typo where I forgot to change the second `self.key` to `self.value`. The type of bug speedy vim programmers make all the time. Because we’ve assigned `key` already, the borrow checker will guard against this kind of behavior. Neato!

You’ve seen `structs` and `enums` before. But not like this.

Instead of telling you, why don’t I show you. For our Redis clone, I want to handle 3 commands (I call actions):

1. GET: get a value

2. SET: set a value

3. INCR : increase a numerical value by 1

Since only one user action is possible at a time, let’s create an Action enum:

pub enum Action {
    GetAction(Get),
    SetAction(Set),
    IncrAction(Incr),
}

We can actually combine an enum and struct together in a way I haven’t seen in other languages. In our case, when we execute the GET command we input something like “GET coffee_shop_name”. Or we might execute “SET employee_count 12”. Each action has some associated data with it. Let’s use structs to define our 3 actions:

pub struct Get(pub String);  // GET key

pub struct Incr(pub String); // INCR key

pub struct Set {             // SET key value
    pub key: String,
    pub value: String,
}

Great, now we know the action the user is doing (like “SET”) and we know the value for that action (key and value). Now, how do we actually “do” stuff using this action. That’s where traits and implementations come into play.

We want every Redis command to follow the same blueprint. In most languages we’d have an “Action” class and then each action (like SET or GET) would be a subclass of that. In Go we would have interfaces. In Rust we achieve this by having traits. Here’s the trait that defines the Redis action:

pub trait ActionTrait {
    // Execute mutates the data
    fn execute(&self, data: &mut DataType);

    // Print the final response
    fn print(&self, data: &DataType);
} 

The trait requires the implementation to have two methods.

1. An execute() method that gets passed a mutable data object. This is where SET will add to the data or INCR will update the numerical value by one.

2. A print() method that prints some result of the command. Some commands return “OK” other’s (such as GET) return an actual value. We could’ve had just one execute() method but I decided to split the “do stuff to data” and “return stuff to the screen” logic. It’ll help with the later WASM section.

Now, we want to implement the blueprint for the SET command. Here’s how we do it:

Incredible stuff. First we’ll run execute() to add the key/value to our data data store, then we’ll run print() to print out “OK” (which is what Redis does when you SET a value). I’ve placed the files for the actions in their own dedicated /actions folder. Can you guess what the GET command’s execute() method does?

The `match` statement:

Now that we have a folder full of command action files, we need a way to run them. Let’s break it down:

1. User passes in a string like “SET coffee_shop_name Sunrise Coffee”

2. We parse the user input string into an Action

3. Based on the Action, we execute and print the results.

Here’s the code for these steps:

One thing Rust does really well is: correctly handling result. Quite often we write code where we either return a value or we return an error. Different languages handle this differently. Rust handles this using the Result type. A Result consists of some data you return via `Ok(value)` or you return an error via `Err(E)`. Simple.

Now let’s walk through the above code:

1. First we’re parsing the Action from the user string using a utility function I wrote called parse_action_from_user_string()

   1. Notice how the return type is a Result. Either the user’s input is a valid action, or we’ll return an error.

2. Next we use a match statement. Just think of it as a switch statement on steroids (I’ll expand on this shortly).

   1. In the OK case we correctly parsed a valid action from the user provided string.

      1. Next we have another match statement. This one matches on the specific actions (GET or SET or INCR).

         1. Either run action.execute() or action.print() for GET, SET, INCR. Notice that there is no execute() called for GET because the GET command doesn’t actually change any data. It just prints a string.

   2. For our root match statement we also have an Err arm for when the user provided string didn’t yield a valid action. Maybe the user input asked for an action we haven’t implemented yet or there was a typo in the string. Who knows.

Stellar! That’s basically the CLI implementation of the Redis CLI core.

I do want to mention that the `match` expression operates in a much powerful way compared to a JS or Python switch statement. Let’s consider this TypeScript code that uses a switch statement (playground):

Notice how the switch statement only considers one case (the MouseDown case). We completely disregarded MouseUpOutside and MouseUpInside. TypeScript doesn’t have an issue with this.

With `match`, every case of the enum needs to be accounted for. Think about how great that is: if I want to add a new command, I add it to the Action enum and at build time the Rust compiler will tell me that there are forks in the code I have to update.

Also, notice how the Result type is being matched against at the root. This is like being able to switch on errors. In other languages we’d need to have some kind of a try/catch mechanism. Oh yeah, and good luck properly propagating the caught error to your observability tool like Sentry. Also guess what happens if I delete the Err arm of the root match statement?

That’s right, a compiler error. This means that match combined with Result forces you to always consider the path with errors. Every JavaScript developer is shrieking.

Compiling executables for the web?

Sounds like a fever dream.

Web-assembly (or WASM) could probably be it’s own blog post (wink wink). The one thing I’ll say is that the WASM + Rust toolchain is really solid. I used wasm-pack to build the WASM binary artifacts. Copied them over to my frontend’s src/pkg folder and used it in my React component right here:

// Import the artifacts from wasm-pack:
import * as wasmPkg from "../pkg";

// Init the wasm class:
const wasm = useMemo(() => new wasmPkg.WasmRunnerContainer(), []);

// Run the run method on my class:
const output = wasm.run("SET coffe_shop_name Sunrise Coffee");

This is powerful stuff. We’re now taking user input on a website → passing it to a compiled module written in Rust → returning a string we show again on a web app.

Some parting thoughts.

So, should we create a JIRA ticket to re-write the entire company in Rust? Well there’s probably some wisdom in the I love building a startup in Rust. I wouldn’t pick it again article.

I think Rust is really impressive. It’s both challenging and fulfilling to learn. The ecosystem and developer experience feels top-of-class. It has shown me the light of a brilliantly engineered typing system. But it comes at the cost of moving slow and a steep learning curve. If you are bootstrapping your own startup or work at a “intense and fast paced” company where changes should’ve gone live yesterday, then stick with more forgiving languages. Tradeoffs to consider. Ultimately it’s your call. I’m just a boy with a Substack.

Also, Jon Gjengset’s YouTube channel and his book are an incredible resource and part of the reason I wanted to learn Rust in the first place. He has the best “advanced” programming YouTube content out there IMO.

There are many things such as macros, fearless concurrency, the module system, and advanced types such as Box and RefCell which I simply didn’t get into. Maybe next time 🤷🏿.

Subscribe now

Share

Edit: if anyone has a “I <3 the Rust match Statement” t-shirt please get in touch.

This post is the first post in the series that goes in depth about building MiniGPT.

The other day I read a blogpost from Cloudflare titled “Writing poems using LLama 2 on Workers AI”. The blogpost shows how using workers AI (Cloudflare’s serverless lambda functions) we can very easily spin up endpoints that interact with LLMs (large language models). The blogpost demonstrates how in roughly 9 lines of Javascript you can send a prompt to an AI model.

The magic happens here:

import { Ai } from "@cloudflare/ai";
...
const response = await ai.run("@cf/meta/llama-2-7b-chat-int8", body);

All the complexity is abstracted away in the “ai” package.

This got me thinking, how hard would it be to make a little ChatGPT application that uses Llama?

A weekend side project

I like to use Notion to track my little side projects. The only way to complete a side project is by tackling the hardest part first. It’s easy to get distracted by the things you’re already good at. If you can get over the initial challenging part, it’ll give you enough velocity to get to the finish line. In this case, I was relatively unfamiliar with Cloudflare workers and interacting with Llama, so that’s where I decided to start.

Using serverless AI functions on the edge

So many techy buzz words! Let me explain:

1. We’re gonna use the @cloudflare/ai package to run our AI function

2. These AI functions (called workers) are distributed to the Cloudflare edge network. It’s impressive how quickly I hit save on the function was available on prod.

⚠️ When launching a new worker AI function, Cloudflare makes it quite clear that this is a beta feature not intended for production traffic. We’re gonna scroll past that warning so fast, it’s like it was never even there.

After going through the Cloudflare worker creation wizard, you have the ability to “Quick edit” the code right in the browser. No need to clone any git repos. The Cloudflare worker editor has a built in textarea (powered by Monoco which is basically VSCode for the web) and has a mini “Postman” area where you can make web requests to your function. Super easy to use.

Here’s the code for the function that uses Meta’s llama-2-7b-chat-int8 model.

import { Ai } from './vendor/@cloudflare/ai.js'
import { handleRequest } from './cors.js';

async function miniLLM(request, env) {
  const body = await request.json();
  const messages = body.messages;
  const ai = new Ai(env.AI);
  const response = await ai.run("@cf/meta/llama-2-7b-chat-int8", {
    messages
  });
  return response;
}

export default {
  async fetch(request, env){
      return handleRequest(request, env, miniLLM);
  },
};

The shape of the incoming message POST body is a conversation between the user and the system:

[
  {
    "role": "user",
    "content": "What is climate change, but tell me in a funny way?"
  },
  { "role": "system", "content": "Climate change is not lit, ..." }
]

This is how most LLMs describe their back-and-forth conversations with the users.

Even though I was able to test out my function in the Cloudflare editor, I was unable to make fetch() requests to it from my local frontend. It was a classic CORS issue. Here’s the code that’ll give you the ability to whitelist specific domains to your worker function:

const corsHeaders = {
    'Access-Control-Allow-Headers': '*', 
    'Access-Control-Allow-Methods': 'POST',
    'Access-Control-Allow-Origin': 'https://gpt.shahzeb.co',
  };
  
export async function handleRequest(request, env, cb) {
    if (request.method === "OPTIONS") {
      return new Response("OK", {
        headers: corsHeaders
      });
    } else if (request.method === 'POST') {
      const res = await cb(request, env);
      return new Response(JSON.stringify(res), {
        headers: {
          'Content-type': 'application/json',
          ...corsHeaders
        }
      });
    } else {
      return new Response("Method not allowed", {
        status: 405,
        headers: corsHeaders
      });
    }
  } 

Not including the CORS helper function, we can now make POST requests to an endpoint which is globally distributed. All in exactly 18 lines of code.

Prompt Engineering 101

I’ve been trying to burn down my Letterboxd watchlist. One movie that’s been on my list forever is the 2005 movie Pride and Prejudice starring Tom Wambsgans from the hit HBO drama series Succession. It’s a solid movie with some really pretty writing (shout out Jane Austen). This got me thinking; can I force the LLM to respond to me as a specific author?

Welcome to a quick prompt engineering crash course. Before we make a request to our AI function, we are going to modify the message we send to the backend with some additional text.

On the UI when a user selects Jane Austen as the desired voice, we change the prompt before making the request. If the user prompts “What is the best drink?” that then becomes: “In the style of Jane Austen, answer the following prompt: What is the best drink?” It’s really that simple. Functionally speaking, typing in that pre-prompt into the prompt textarea and selecting Jane Austen from the dropdown will get the same result.

Here’s a snippet from the frontend for of how we convert the prompt voices:

export type voices = ("none" | "brevity") | authors;
export type authors =
  | "Jane Austen"
  | "Ernest Hemingway"
  | "John Steinbeck"
  | "Mark Twain";

export const PROMPT_VOICE = (prompt: string, voice: voices): string => {
  switch (voice) {
    case "brevity":
      return `Within 4 sentences or less, answer the following prompt: ${prompt}`;
    case "Jane Austen":
      return `In the style of Jane Austen, answer the following prompt: ${prompt}`;
    ...
    default:
      return prompt;
  }
};

I added the “Brevity” option as a voice because the shorter response means the AI runs faster and is less likely to end on an open sentence.

In our client, where we make the request to our backend, we use this PROMPT_VOICE function to reshape the prompt in our userInput:

const userInput: Response = {
      content: PROMPT_VOICE(prompt, promptVoice),
      role: "user",
    };
const req = [...prevResponses, userInput];
const res = await promptRequest({ messages: req }); // does the fetch()

At this point we have a functional deployed backend we can have a conversation with.

I hate to leave you on a cliffhanger…

…but this post is getting very long. Substack is yelling at me that this post will not fit in an email. Whole time I’m thinking, who reads email.

Hopefully you do because if you subscribe (it’s free, just put in your email bellow) I’ll be sharing two new posts in this series:

1. The frontend for MiniGPT

   1. Tools used to build it

   2. Deploy it

   3. How to debug the most pain-in-the-ass mobile responsive UI bug with a CSS hack

   4. ‼️ Link to the code ‼️

2. The best side project database I’ve found

   1. I’m using it to track the total number of prompts so far on the homepage

   2. Avoid AWS at all costs

   3. Spend like $5 on it

As always, thanks.

I was an early user on Svbtle back when I saw a “Show HN” thread about it on HackerNews.

I loved the minimal design. I loved how it works really well on mobile. I dig the draft preview URLs. I think the composer makes it super easy to write without distractions. I’ve had no complaints about it for the past ~4 years. It does what every blogging platform should do, without any-frills.

The main reason I’m moving on is: SSL. Chrome would show a privacy error when users visited my custom blog domain because Svbtle doesn’t use custom domain SSL certs. Custom domain certs have been a private beta feature for more than a year now. I’ve written to the Svbtle team asking them to give me access, but have gotten no response from them. On Chrome (and any browser using a “Force HTTPS” extension) this makes my blog totally unusable. Having custom domain certs for paying customers should have been treated as an urgent high priority item. It should have been on by default for all users.

Anyways, I’m sad to move on from Svbtle, but excited for a fresh start on a new platform. I’ll be writing about random AI / web things I’m learning and building. You might get a post or two a month. Feel free to subscribe (it’s free and all the cool kids are doing it).

Subscribe now