This week (thirty years ago) one of the all time great albums hit #1 on the charts in its first week of sales. It would go on to sell over 10 million copies, making it one of the best selling albums of all time. And deservedly so. Mellon Collie and the Infinite Sadness by The Smashing Pumpkins is one of my favorite albums. According to last.fm, I have played a song off the album over 600 times.

I remember when the lead single off the album, Bullet with Butterfly Wings, hit the radio. Of course it's a banger, but what I remember thirty years later is mishearing the line "despite all my rage, I am still just a rat in a cage." I could make out the "despite all my rage" part, but I couldn't quite figure out the second half. I think I had some nonsense words there, but it's been so long since I learned the correct words that I've forgotten what I thought the words were!

Another single off the album, 1979, was cool because that's the year I was born. The song is about entering adolescence, and in 1995 I was in the throes of adolescence myself. A great coincidence!

All of the big singles off this album still get plenty of radio plays, but the whole album deserves to be listened to in its entirety. I did not find listening to this album another time for this project a chore or unpleasant. Indeed, it was a pleasure, and I look forward to listening to the album again and again for years to come. You should listen to the album today, and again and again.

Last month the company I work for purchased a subscription to Augment Code for all its developers. Augment Code is an AI coding engine that is broadly similar to Claude Code, which I played around with a few months ago. You can read the linked post, but the summary is that I came away mostly skeptical about AI coding. However, I am not a luddite, and am willing to learn new tools and try things again, so I installed the Visual Studio Code Augment plugin and have been giving AI coding another shot.

What I've learned is that AI coding agents are more useful than I gave them credit previously if you give them small enough jobs that are well-defined. Asking it to do too much, which perhaps I did in my earlier blog post, is not (yet) what it's good at. Augment code has a few modes. There is a chat box in which you can conversationally interact with the AI, asking it questions or giving it tasks to complete. The agent will also give autocomplete suggestions as you type new code, which I'd say is mostly helpful, but not always. Sometimes the suggestions are just plain wrong, but a few times the suggestions have been subtly wrong, which is dangerous.

Here are some example tasks that Augment Code has been at least 90% successful at:

• In one of my Python projects, I asked it to create a file that tries to import all the packages used in the project, and output which packages do and do not import successfully. This project uses a few custom & private packages I keep elsewhere, so a requirements.txt file doesn't work with pip. Having a file I can run to quickly check I installed all the packages is useful. It did a pretty good job of this, but it did miss one import from one file, probably because the import wasn't near the top of the file (which is an anti-pattern, but it's there for reasons)
• It is quite good at adding type hinting to Python projects. You can ask it to "add type hinting to all functions in all Python files in this directory" and it does it
• I have a PyO3 project that it successfully threaded/parallelized using Rayon. I had to be very specific about how the inputs and outputs would change, but with that it did a good job. It wasn't perfect. Instead of using lightweight vector slices, it was creating new vectors for each chunk of parallel work, which involves copying memory. When I suggested a change, it did a good job fixing that oversight
• I am an unapologetic user of Mercurial. The rest of the world uses Git. Kind of like Mac and Windows, Mercurial and Git are 99% the same in what you can do with them, but they differ in methods and style. In fact, there is a Mercurial plugin that allows perfect 1:1 interoperability between the two, which I use. Sometimes I don't want to bother installing Mercurial on a temporary virtual instance that already has Git installed, and I want to do something quick in Git. I've asked the AI agent to translate a Mercurial command to Git and it's done a fine job

It appears that my company has a (grandfathered) $50/mo/user plan, which has jumped to $60/mo/user now for new purchases. I would say that it has saved me enough time to justify that price. The real question is how much the service actually costs? The AI industry is spending so much money that "to recoup their existing and announced investments, AI companies will have to bring in $2 trillion (every 2-3 years), more than the combined revenue of Amazon, Google, Microsoft, Apple, Nvidia and Meta." It feels like the early days of Uber, where the fares were subsidized by venture capital, and once most competitors were vanquished, prices went up. To reach $2 trillion in revenue, how high would prices have to go? There are currently a bit over 8 billion humans on earth, which means that over 3 years, the whole AI industry will need to take in about $80 from each and every person on earth per year. That is a ridiculous number and will not be reached any time soon.

My opinion is that AI does has some value, but not nearly as much as it costs in real terms. I'll use it if it works for me. But I won't rely on it.

Rising to #3, Design of a Decade 1986/1996 is a greatest-hits album of Janet Jackson.

When it comes to the music, I don't care about it. I've heard a few of the songs before, likely contemporaneously, and also since 1995. Janet Jackson had been a big star for years, so of course I've heard some of her music. It's solid pop music, I concede, but it's not for me.

What I find more interesting is the name of the album: Design of a Decade 1986/1996, what does that even mean? I do not believe that Janet Jackson planned (designed) a whole decade of music, recorded it, with the target of a greatest hits collection. Also, the title is factually wrong! The album was released in early October 1995, nearly three months before 1996 even started. 1986 to 1995 represents ten years (aka a decade, count it out). Putting 1996 in the title is both illegitimate and numerically incorrect. The only thing it has is alliteration, which isn't much (lots of words start with the same letter as other words). It's a stupid name. Stupid!

My verdict: This album can be left to the dustbin of history.

After last week's album, some Green Day is very welcomed. Coming off their smash hit Dookie, Insomniac was return to harder, less poppy punk music. Debuting at #2, it never reached #1 and didn't sell anywhere near as well as Dookie (although few albums do).

I don't know why but I haven't listened to Insomniac that much over the years. I do like punk music, and listening to it now it's in my wheelhouse of musical taste. I think I'll add it to my rotation and I'll try to revisit it more often. This is a win and why I do this project. I suggest that you check out Insomniac, too!

Sigh, it's another Mariah Carey album. I'm pretty sure this is the third Mariah Carey album I've done. This is gonna hurt.

Rocketing to #1 upon release, Daydream had three singles reach number one on the US Billboard Hot 100 list. Because of the popularity of the songs off this album, I certainly recall hearing them on the radio.

I don't miss hearing the songs as frequently on the radio, and I didn't appreciate hearing them again. I get that Mariah Carey is very popular and successful, but I will be happy to never review one of her albums again. It's not for me.

Ballbreaker by AC/DC debuted at #4 on the album sales chart.

There's nothing particularly special about this album. None of the songs are amazing, the lead track Hard as a Rock was the best performing single and only hit #33 in the US. That being said, none of the songs are terrible, which across a whole album is somewhat unusual. That's probably because the band had over 20 years of experience.

As far as it goes, the market agrees with me. Next week the album drops to #9, then #12, and so on. This is a common pattern with older, established acts on the charts. They sell well to start but there's no long tail of people discovering them keeping the album sales high.

If you're into AC/DC, check it out, but don't expect much.

Debuting at #4, All I Want is the second Tim McGraw album I've reviewed. I closed my previous review with the sentence "Finally, I will not be listening to this album again," and I have stuck to that promise.

In a very real sense, All I Want is really no different than Not A Moment Too Soon. The second line of the first song is "Finally own a car that doesn't break down on the freeway." He wasn't pushing any musical boundaries with this release. He had a formula that worked well 17 months prior, and he stuck to it. I'll begrudgingly admit that the biggest hit off the album, I Like It, I Love It, is good fun.

Overall, my opinion of this hasn't changed, and like before, I will not be listening to this album again.

It's been a while since I reviewed an album by any artist(s) I actually like, therefore reviewing a Red Hot Chili Peppers album is a big relief. As of this writing they are #8 in my rankings of listens by artist.

One Hot Minute (the #4 album this week) was released four years after RHCP's best album, Blood Sugar Sex Magik, which was always going to be a tough act to follow. One Hot Minute has never been my favorite RCHP album. It isn't bad at all, but compared to Blood Sugar Sex Magik it was always going to pale. Interestingly, the internet ranks One Hot Minute all over the place, from worst, to fourth, and spots in between. I'm not knowledgeable enough about the RHCP corpus of work to say where One Hot Minute fits.

There were a few decent hits off of the album, including Warped, My Friends, and Aeroplane, all of which I remember hearing contemporaneously on the radio, and sometimes still do on "classic rock" stations (a thought that makes me feel old!).

This album is probably worth your time, as are all RHCP albums.

I think there is only one song off of this week's #9 album by Silverchair that I've heard before. Tomorrow off of Frogstomp got a decent amount of airplay when it came out. The rest of the album seemed new to me, and my last.fm history pretty much confirms it.

Silverchair definitely rode the tails of the Grunge wave, and they did a good enough job of it to sell a couple million copies of Frogstomp. I don't think that I will increase my Silverchair consumption a great deal, but perhaps I'll think it more often when I'm in the mood for some Grunge. You might consider the same!

I am a big proponent of Polars for data analysis. It combines the ease of Python with the speed of Rust. In many cases it allows working with very large datasets very convenient. I have even written about it before on this blog.

Despite the nice things I wrote above about Polars above, it is still a work in progress. One big issue it has is less than ideal memory management when working with very large dataframes as discussed in this ticket on github. I have seen similar issues with Polars, where for no explicable reason it will run my machine out of memory. Below is a simplified example of a situation that I have run into this problem.

This example is in the middle of a processing pipeline. I have created a set of Parquet files in which I am including an ff column because I want to be able to filter on it. The entire set of Parquet files in dirname would not fit in memory, but a subset will, and I can process these subsets separately and combine the reduced dataframes at the end to achieve the same result. The distribution of values in ff is very even, meaning that each subset uses very closely the same amount of memory. Because ff is in the Parquet files, Polars can use predicate pushdown to filter on ff at the read step, meaning that the full dataset is never read into memory.

Indeed, the first few loops of this process work quite well. The filtered subsets fit into memory, and reduced dataframes are calculated. However, eventually Polars stops managing memory correctly and things go bad, and the machine runs out of memory.

import polars as pl

data = pl.scan_parquet(dirname)

reduced = []

# this will eventually run out of memory
for ff in range(N):
    print("doing", ff)
    # note that data is untouched each cycle
    data1 = data.filter(pl.col("ff") == ff)
    # Do some various LazyFrame manipulations on data1; we don't call
    # .collect() until below
    reduced.append(data1.collect())

reduced_data = pl.concat(reduced)

If the situation is amenable, there is a way around this. By running the Polars steps in a subprocess using multiprocessing, we can force Polars to free memory after each cycle by shutting down the process that runs Polars every cycle. Of course, there is a little bit of additional overhead here. Starting the subprocess takes a little time, as does re-scanning the directory of files in scan_parquet(). There is also some cost in un/pickling the reduced dataframe between the main process and the Polars subprocess. But if the data is large enough to resort to chunking like this, these extra costs are rounding errors, and are not worth worrying about.

from multiprocessing import Process, Queue, set_start_method

def worker(dirname, ff, out_q):
    data = pl.scan_parquet(dirname)
    data = data.filter(pl.col("ff") == ff)
    # Do some various LazyFrame manipulations on data; we don't call
    # .collect() until below
    out_q.put(data.collect())

if __name__ == "__main__":
    set_start_method("spawn")

    out_q = Queue()

    # this works to completion
    for ff in range(N):
        print("doing", ff)
        p = Process(target=worker, args=(dirname, ff, out_q), daemon=True)
        p.start()
        reduced.append(out_q.get())
        p.join()

    reduced_data = pl.concat(reduced)

I think this trick very simply illustrates the way that Polars is currently mismanaging memory. The two examples above are doing the same basic thing, but one works and the other doesn't. In fact, if I run the two methods above and watch memory use in real time, the two methods have similar behavior for some number of cycles. The memory use goes up and down as the filtered data is processed and cleaned up each cycle. This indicates that Polars can do memory management correctly, but for some reason after too many cycles, Polars stops freeing memory. In the github ticket linked above, it's suggested that a fix might have a performance cost. Hopefully something can be done about this that both prevents the memory leak and doesn't cost too much in performance. For now, this trick can be useful in the right situations.

It's been a busy week, and I am five days late on this review. Luckily I don't have much to say about this week's #8 album. Apparently there were a couple hits off of Conspiracy by Junior M.A.F.I.A., but listening to the album didn't jog any memories. I am not disappointed because I didn't find the album terribly entertaining. I cannot endorse listening to this album, and I will never listen to it again.

There are two albums in the top 10 this week that I have not reviewed. At #4 is a compilation album The Show: The Soundtrack, and at #10 Games Rednecks Play by Jeff Foxworthy. I can find neither of them on any streaming service.

The Show was a documentary about hip-hop, and the album features on the order of 20 artists. Likely due to various byzantine licensing issues with so many artists, putting the soundtrack on a streaming service would be next to impossible.

Games Rednecks Play is a comedy album, and comedy has had many licensing issues with the online music streaming services. It's not surprising that I can't find it.

Because I want to review only popular albums from 30 years ago, I don't drop below the top ten. Therefore, I have nothing to review this week.

If you were alive in 1995, you heard the song Gangsta's Paradise everywhere. The popularity of this song propelled the soundtrack to the movie Dangerous Minds to #1. The song made Coolio's career, although playing Kwanzaa-bot on Futurama was pretty great, too. I'm fairly certain I have never seen the movie Dangerous Minds, but I have heard Gangsta's Paradise many, many times.

None of the other songs on the album amount to much. I guarantee that you would not recognize any of the non-Coolio artists on the album. Gin & Juice by DeVanté is far, far inferior to the Gin and Juice by Snoop Dogg you're thinking of. A Message For Your Mind by Rappin' 4-Tay samples I Want You Back by The Jackson 5 and completely misses the mark, somehow having none of the joyous energy of the original song. The final song (which I suspect was played during the closing credits), This Is The Life, is a non sequitur. It's a ballad by two white women who were part of Prince's The Revolution band. There's nothing wrong with being female, white, or working with Prince, but none of those things match anyone nor anything that came previously on the album.

In summary, the album is defined by and entirely worth the value of Gangsta's Paradise, and the rest is worth forgetting.

Finally, check out this episode of Last Week Tonight with John Oliver from a few weeks ago. It's about law enforcement gang databases, and a connection to Gangsta's Paradise comes out of left field.

Rising steadily in the charts, the #3 album this week will hit #1 in a few weeks. I'm not sure I have ever listened to the entirety of Jagged Little Pill by Alanis Morissette; certainly not since I started tracking plays on last.fm. I have, of course, heard all the big hits off the album, of which there are many. Jagged Little Pill was huge when it came out and You Oughta Know was everywhere on the radio in 1995 and 1996. The non-ironies in Ironic have been pointed out for decades. The album might have contributed to the English language. It's possible that the term "Friends with Benefits" originated in Head over Feet.

It's hard to form any new opinions about this album because the singles were so ubiquitous. They were earworms then, and they are earworms now. There's no harm in listening to the album, short of a Friends marathon, it's one of the best ways to transport yourself back to (or get a small taste of if you're too young) the mid-90s.

The #4 album this week is the solo debut of Raekwon, a founding member of Wu-Tang Clan. Like the last album I reviewed by a Wu-Tang Clan member, my summary is that while I recognize that Only Built 4 Cuban Linx... is an important and well-regarded rap album, it's not my preference. I will almost certainly never listen to it again.