Embracing My Inner Blub Programmer

I always feel a little bit guilty about Lisp. I am supposed to appreciate Lisp. It’s a fine wine and as a member of respectable society I should hold it in high esteem. But, I’m not quite there. I’m not opposed to Lisp, per se, I just never end up using it for very much.

This week is no exception. I am not writing any Lisp this week. Quite the opposite - for the first time in a while, I am writing big chunks of C++.

The Bad

It’s very obvious when you write C++ after using languages like Python or JavaScript for a while that C++ has problems. Ancient problems that I remember Java fixing back in the 90’s. The problems with C++ are so glaring, I run into programmers who are surprised that such things are even possible.

• When you write a new class you have to split logic into .h and .cpp files, half repeating yourself.

• If you accidentally access of the end of an array, your program will simply crash with no error message reported.

• If you forget to initialize a member variable, your program will often set those variables to zero, but sometimes it will just fill them with quasi random data.

• You either have to manually free memory that you allocate, or use solutions like unique_ptr that are more complicated than any reasonable language offers.

The Not So Bad

At least C++ is getting better. The last time I wrote a lot of C++ was working for Google, 2005-2009, and I don’t think we had even fully adopted C++98. So I’m still a decade behind the times, learning the various C++11 things. unique_ptr and shared_ptr are good; they let you mostly avoid remembering to delete your own pointers. mutex and condition_variable are good as well. And in general with GitHub, cmake and meson there are a decent amount of libraries out there that do common things. Not like Python or JavaScript where you have solutions for everything, but it’s a lot better than nothing.

The Actually Good

So why C++ at all? Well, this is my favorite API of the week:

cublasCgemm3mStridedBatched(cublasHandle_t handle, 
                            cublasOperation_t transA, cublasOperation_t transB,
                            int M, int N, int K, 
                            const T* alpha,
                            const T* A, int ldA, int strideA, 
                            const T* B, int ldB, int strideB, 
                            const T* beta,
                            T* C, int ldC, int strideC,
                            int batchCount)

It’s a shortcut to run code equivalent to these nested loops, on a GPU…

for (int p = 0; p < batchCount; ++p) {
  for (int m = 0; m < M; ++m) {
    for (int n = 0; n < N; ++n) {
      T c_mnp = 0;
      for (int k = 0; k < K, ++k)
        c_mnp += A[m + k*ldA + p*strideA] * B[k + n*ldB + p*strideB];
      C[m + n*ldC + p*strideC] = 
        (*alpha)*c_mnp + (*beta)*C[m + n*ldC + p*strideC];
    }
  }
}

…but you can also conjugate-transpose A before running this matrix operation. I was pretty excited, I literally wanted a matrix to be conjugated and transposed, and I thought this was the sort of abstract mathematical fun that I had given up long ago with the transition to computer science.

See this Nvidia blog post if you’re curious for more detail, but basically this code is multiplying sequences of matrices that you’ve stored in some regular way in memory.

Blub

I feel like this is why I always end up developing in Blub nowadays. I’m never choosing programming languages based on the most powerful generalist language. I always have some narrow task - from GPU matrix operations to making a responsive website where you can drag stuff around - and having a library that’s well suited to the task ends up being more important to me than having macros.

It’s not the same but at least I sort of get to use Lisp macros while developing in Emacs. I’ll take another look at VS Code once they implement C-x-(, C-x-), and C-x-e.

H1 Review

About six months ago I made some New Year’s resolutions, and I thought I might write a Q1 update, but here we are six months later and I haven’t written anything about how it’s going. So… it’s an H1 update.

1. Calorie Counting

Man this one is going well. This is the most effective diet I’ve ever been on, and I’ve been sticking on it for six months. It’s pretty simple to describe - I just count calories, using the default iOS health app. I am actually below the official government “not overweight” line for the first time in my adult lifetime.

So, yeah, great, I just need to keep it up.

It’s somewhat less simple in practice than it is in theory. It’s really not straightforward to figure out how many calories are in a given food. Even the best information on the internet will not be accurate to within about 20%. The “counting” isn’t the hard part, it’s the “estimating calories for a given food item” that is the source of errors.

I also just haven’t been drinking very much. Two days this past six months.

2. Astronomy Publication

I don’t think this goal is on track. Six months ago I thought it was plausible for my work on processing the Green Bank data to get rolled up into some summary paper, soon, describing our work on SETI search there. But, it’s looking like there isn’t a critical mass of people who want to work towards publishing a big analysis of Green Bank data this year.

I’m shifting the strategy here a bit. I’m focusing on this new repository of SETI-search and general radio astronomy algorithms, with two main principles.

1. Faster than any alternative, via writing in C++ and CUDA
2. Full support for interferometer arrays

My goal is to get this new software adopted as the production SETI search library for both MeerKAT and the VLA by the end of the year. These are interferometers rather than big single dishes, these are the new hotness, I should really write some separate blog post on just “what are interferometers”.

I’m less focused on “what algorithm exactly is it running”. I have a lot of ideas for algorithmic improvements, but everyone has a lot of ideas for algorithmic improvements. There are dozens of cooks in this kitchen. The limiting factor is the ability to implement search algorithms, and to implement them efficiently enough to actually run them on the scale of terabytes-per-minute.

So, if there arises a consensus in the scientific community that algorithm X would be a fantastic way to analyze incoming radio telescope data, I’d be interested in writing an efficient implementation of X, and integrating that into seticore.

I’m also less focused on “publication”. That’ll come in time and I don’t really want to work on a publication just for the sake of getting a publication - there’s too much of that in academic-world already.

3. Exercise

I’ve just been cranking along here. Working out four times a week is a pretty regular part of my routine now.

It’s funny, I don’t really think exercising has helped me lose weight. It just doesn’t seem like it correlates. But the reverse is definitely true - losing weight helps me exercise better. I have somewhat more endurance, and way more ability to do pullups.

I do think that exercising gives you more energy through the next day. Same with not drinking. My energy level the next morning is really boosted when I exercise and when I don’t drink.

4. Blogging

I’m a little behind on this one. Just a couple weeks behind, though. I wrote a script to compare how I’m doing.

I used to blog and try hard to get people to read it. I just don’t have the mental energy to do that every week. And it’s a totally different blog post. The stuff people want to read is like, zingers about Uber when Uber has been in the news. Thumbnails of GPT-3 which are easy to exaggerate.

The stuff I want to write is like, I read some books about the history of Mexico, now I have all this information swimming in my brain, and I feel like I should do something with it. I need to get it out, need to stop just thinking random uncorrelated thoughts on the topic and see if I can boil down what I really think. I want to reflect on the Drake Equation or AIs playing board games or the importance of slug welfare.

So, I am just not going to try to get people to read this. Not now, at least. But I’m going to keep writing. I’ll try to catch back up - I fell behind when I caught Covid a couple weeks ago.

Looking Forward

I’m feeling pretty good about the whole resolution thing. Calorie-counting has been a great success, astronomy publication has not been a success but on a meta-level I feel like the explicit goal-setting process was useful there because it made me think a bit harder about what to focus on.

It’s funny because at Google and Facebook I really hated the performance review cycle. One of the worst parts of working for a big company. But I like the process of having goals and being tough on yourself for not hitting them. I just hated being reminded that I was in a system where some other person’s opinion of my performance was really important to me. Every single time it would make me think, I wish I wasn’t working here. I should quit. I should be testing myself against what I can achieve in the world, not against what some coworker thinks.

Onward to H2. I’ll aim to do a similar writeup at the end of the year.

Book Roundup: Native Americans

Today I visited the Makah Indian Reservation, at the most northwestern point in the continental United States. I was watching a group of distant cormorants with binoculars, and slowly realized that two of them were actually puffins.

There are several reservations scattered around Olympic National Park, and it’s interesting to see what they are like nowadays. I only get a tiny glimpse, though. It is a bit of an inspiration to read about indigenous culture.

What I most wonder is… could history have gone differently? How else could it have gone, other than this clash of ways of life, this quasi war of civilizations followed by the mistreatment and sidelining of native culture? It’s one of those things where I read looking for the answer to a question, but I just find more questions.

The True History of the Conquest of New Spain

I read this book a while ago, but it has stuck with me. This is a narrative account written by one of the guys who was fighting alongside Cortés as they conquered the Aztec empire. It’s just so alien. Not once does he stop to wonder, hey maybe killing all these people, burning and destroying their religious things, invading and destroying their country, maybe we’re doing a bad thing here. No, all of that is taken for granted. Death and destruction, one thing after another, risking his own life, dozens or hundreds of his companions dying, but obviously it’s all for a good cause.

Not only that, but the author has an extremely strong ethical sense, and is constantly arguing about what’s right and wrong, morally. He is just completely not focused on the natives. He’s upset that other groups of Spaniards either don’t give him enough credit, didn’t give him the cut of the spoils that he deserved, or included wrong facts in their retelling of this story. There’s so much of this that basically all modern editions cut out a couple hundred pages that steps away from the story to argue about some financial dispute that nobody has cared about for hundreds of years.

Was the Spanish victory inevitable? It’s hard to believe it happened at all, honestly. The Spanish were massively outnumbered, their weapons weren’t all that effective, they didn’t speak the language, but a small group basically managed to kidnap Montezuma and in the resulting confusion parlay that into conquering Tenochtitlan. I’m surprised they even tried in the first place.

I feel like the greatest weakness of the Aztecs is that they just did not know what was going on. They didn’t understand how counterproductive it was to give gifts to the Spanish, they didn’t know that the Incan empire had just fallen, and they didn’t understand that they were facing a threat significant enough that they should stop fighting amongst themselves.

1491

This book made me sad for all the memories we have lost. Was there a corn-farming empire in the American Midwest? Are there complex civilizations buried beneath the Brazilian rain forest? Did the Spanish destroy records written in languages we don’t know anything about any more, full of rich stories and histories of forgotten peoples?

I understand that civilization moves on. I don’t lament that I live in a modern city with technology and medicine. I’ve lost touch with the skills that my ancestors of 400 years ago needed to survive, and that’s okay. But at least we have the stories. We have the language, we have the records, we descendants of Western Europeans know where we came from.

I wish the Spanish had simply collected the artifacts of the religions they hated, instead of destroying everything they could.

There’s just so much in this book. It makes me want to travel around Mexico and Peru.

Lakota America

The Lakota are interesting to me because a high school near where I grew up in Cincinnati was named “Lakota”. Our chess team would play against them. But nowadays the Lakota are around South Dakota. I never really thought about that too much until I read this book.

The story of the Lakota is a surprising one. Usually I think of Native American groups in history as a static thing - they lived somewhere, they had a way of life, the United States expanded, ran into them, probably treated them poorly, maybe there were some battles, and they end up forced onto a reservation. The Lakota don’t follow this pattern; they were shifting their way of life and expanding their own empire over time, and the Americans didn’t quite realize what was happening.

The fundamental shift in Lakota culture was when they got a lot of horses. Hunting buffalo on horseback was so profitable, they shifted their entire economy to this one “crop”, and migrated west in search of the best buffalo territory, often conquering other Native American groups as they went.

So from maybe 1730-1870 this was a great strategy and the Lakota grew to be the strongest Native American group. The core problem was that the buffalo were a limited resource. As the United States expanded, more settlers hunted the buffalo, and the Lakota by themselves were probably overhunting buffalo as well. There were some famous battles at the end, like “Custer’s Last Stand”, and some unethical manipulation of people via “treaties” that the Lakota couldn’t read - but the United States didn’t really win in the end either via military prowess or via trickery. The Lakota were dependent on American trade in a single declining industry, so trade restrictions crippled their economy, essentially forcing them to accept terms of surrender, moving onto reservations.

Was this inevitable? I don’t think so. The Mormons at some point were running a theocratic empire in the middle of nowhere, and we ended up figuring out how to compromise into a happy future, where nowadays Utah is a prosperous state, just like the others, and it just happens to have a lot of Mormons. We could be living in a world where the Lakota and the United States came to a peaceful compromise, and nowadays South Dakota just happens to have a lot of Lakota.

What went wrong? I think the Lakota had to get into some variety of high-productivity farming. They controlled a lot of economically valuable farmland, and the Americans had so many other priorities, like establishing the cross-continental railroad lines. It seems like it should have been clear at some point that war would not solve their economic problems.

Recommendations

The overall best of these books is 1491. It’s well-written, interesting, broad, and leaves me hungry for more on the topic. I recommend that to just about anybody. On the other hand, I feel like the other two went deeper and made me think harder. If one of the others piques your interest, those are pretty good too.

Book Roundup: South Africa

Often I get interested in a particular topic and I read a number of books about that topic. After a while I feel like I have learned something from a few different directions, I get sated, and I start to sort of sum things up in my head.

So I’m trying something a little bit different book-wise. Whenever I have finished a few different books on a topic that I like, I’m going to bunch them together and write something about them. This is useful from my point of view - when I’ve read several good things on a topic it’s the time that I want to reflect, and writing a blog post is a good way to reflect. I’m not particularly convinced that it’s good for you the reader to read my comments on books in this way. But, we’ll see.

I guess this isn’t all that different - I’ve gathered book reviews together before on the topics of Africa, biographies, and mopey books. Perhaps I should say, I like this format and I’m sticking with it.

Today’s book review topic is a narrowing of the last topic - not just Africa, but specifically South Africa.

Life & Times Of Michael K

This book is good in a literary sense. It isn’t my favorite Coatzee - I liked Waiting For The Barbarians better. The main character has this idea that sticks with me, that he is sort of “dissolving into time” as he sits around doing nothing.

The South African regime seems Nazi-like or Stalinesque here. Prison camps, work camps. I feel like still don’t know very many core details about South Africa despite all this reading. This book hits the emotional angle but not really a factual one.

The way you can stake your whole life on basically nothing when you have no other alternative… I don’t have a verb phrase for this sentence, but that thought, this book.

Born A Crime

I rarely read the genre of book that is “popular autobiography of a popular comedian” but the other day I was at a bookstore, picked this one up to leaf through it, and just found it really compelling. He was born to a mixed-race couple when that was illegal in South Africa so his autobiography just gets off to a really exciting start, dodging the law with a newborn.

I like it a lot. You sort of know how the story ends up - he becomes successful in a weird career - although I actually haven’t watched any of The Daily Show since he started hosting. But it’s really just interesting for the stories of South Africa from an unusual racial background.

I didn’t realize just how many racial groups were fairly prominent in South Africa. Xhosa, Zulu, maybe a dozen others. It gives me this feeling of a whole world that I have only barely scraped the surface of.

Triomf

This book is a bit of a crazy one. It was unexpected to me how this and the Michael K book are from the point of view of white citizens of South Africa, and yet the main characters are completely and utterly miserable, suffering with a terrible life, like if these had been in countries where they were racistly kept down it would make total sense, but these aren’t even the people bearing the brunt of the South African regime!

In the US these characters would be “white trash”. In South Africa there is probably some analogous pejorative, I just don’t know it.

The characters are just so, so lost. So far from any happy life outcome. I just can’t imagine an alternative storyline where they end up succeeding. And yet are they really unhappy? I don’t know.

Conclusion

It’s funny I’m reading all these books about South Africa as I road trip around the Pacific Northwest. Often I like to read books about a place as I travel through the place. This is more like doing the opposite.

I must have learned about South Africa in reading these books. But emotionally I feel like my level of understanding of South Africa is lower than it was before. It’s one of those cases where every question you answer, you learn about three more questions that you don’t know the answer to.

My best book recommendation from these is honestly not even one of the ones listed, it’s that I’m reminded of Waiting For The Barbarians which I liked better than any of these. So go read that one. It is kind of about South Africa, too, although it nominally isn’t, and I didn’t realize it when I first read it.

A Trip To Green Bank

I took a trip out to Green Bank recently, home of the Green Bank Telescope, to do some upgrades on the onsite computer systems that process incoming data from the telescope, but also to take a cool tour and learn more about the workings of radio telescopes.

It’s really big! The highest point is about 480 feet off the ground, and the dish is 300 feet across. It feels like going up the Eiffel Tower - you’re usually surrounded by metal frameworky parts as you go up, and it’s hard to take it all in. Imagine a 35 story building, but the top half can rotate around to point at different things in the sky.

Here’s what it looks like from the ground nearby.

This picture is from the next morning - to go up to the top, the telescope has to be in “maintenance position”, where they point it so that the top tower part is vertical, so you can go up the elevator.

Basically, the way it works is it needs two degrees of freedom so that it can point anywhere in the sky. The bottom is built on 16 legs that rest on a circular train-like thing that rotates the whole thing around. Then the “C” shape in the middle is like the edge of a big gear that rotates the dish up and down.

When there’s too much snow on the dish, they just tilt the thing sideways to dump the snow off.

Radio waves come in from places in the universe, bounce off the big dish, and then bounce off the little dish you can see way up at the top, called the “subreflector”. The platform right below the subreflector has the receivers, which are the sensors that measure incoming radio waves.

The teacups are the receivers. The big circle on the floor is like the part on a microscope that you can rotate to use different lenses - it rotates so that you can put different receivers in the beam. Different receivers measure different ranges of frequency of radio waves.

In the room below this, each of these circle things is connected to basically a refrigerator. Sometimes it’s a refrigerator with another refrigerator inside it, with another refrigerator inside that. The sensors have to be really cold because they are so sensitive, they pick up background heat as radio noise. No pictures in that room because of secret stuff.

Here’s a selfie of me standing at the lower level, by the big dish. Yeah I gotta trim my beard. You can just walk out onto the dish. But it doesn’t have a railing or anything, occasionally a panel falls off, and it’s hundreds of feet off the ground. Usually I consider myself “not afraid of heights” but this is the sort of trip that makes you reflect on where your limits are.

Once the receivers measure the radio signals, they output an analog signal, it goes to a datacenter nearby, and then the things that happen there are pretty normal things, from a software engineering point of view. The telescope produces data at a much faster rate than there is bandwidth out of the facility, so you have to do your heavy processing onsite.

It was a fun trip. I really appreciated the opportunity to see the telescope and to have a captive audience of engineers and astronomers that I could pepper with questions.