That depends heavily on the shape of your data, what your workload looks like, what sort of consistency guarantees you need, etc. I recommend Designing Data Intensive Applications for getting a handle on this - it's the book I suggest to SDE IIs who are hungry for the jump to senior. Not a quick read, but well written, and there's not really a shortcut to deep understanding other than deep study.
> I suggest to SDE IIs who are hungry for the jump to senior
Any suggestions for seniors who are hungry for a staff title. Is there any value to specializing in tech anymore after senior or is it all 'soft skills' at this point ?
It is not all any one thing. If you’re completely lacking in soft skills, you’ll probably have more trouble the higher up you go, and you may get stuck at some point. Hard technical skills become less important the higher you go, but even the CEO of a tech company probably still needs to have some basic understanding of technology. Being good at both things is obviously the best way forward.
I write software at a fintech, and the CFPB regulations are super important for protecting the customer. They also give important guidance on how the customer needs to be taken care of, which is important for a company that wants to do the right thing but doesn't have expertise in customer service.
Semi professional musician here. Repetition goes way beyond what the author mentions:
- Most music is highly repetitive, often recycling 2 or 3 short segments (chorus, verse) with minor variations to fill out a whole song. Coltrane is known for his avant-garde composition, and even he repeats (often on a much smaller scale than a pop tune).
- The work of being a musician is repetitive. Learning (memorizing) songs takes reps! Then you've got to keep them fresh, teach them to new band members, etc. You probably have a limited book, and you know what the crowd pleasers are. Unless you're big enough to have a following cutting a song you're sick of isn't a problem, but filling out a set might be. Between rehearsal, gigs and practicing at home I probably play through most of my band's book at least twice a week.
- Being a musician is very physical, which means you're drilling exercises in your daily routine. As a brass player, I run more or less the same set of warmups, range builders and flexibility exercises every day. Drummers do rudiments. String players have their own shtick.
As far as listening to music, I don't typically put something on repeat unless I'm trying to transcribe it. But I'll listen to a song, and there's a chance it'll play on repeat in my head all day (or all week!). Steely Dan and LCD Soundsystem are particular earworms for me. It wasn't until college I realized this isn't true for many people.
I once had a gig playing piano at a restaurant. It was a weekday mid-day stretch and I was scheduled for about 3 or 4 hours, as I recall. I put together about a 90 minute set of stuff out of my fake books that I figured I could do without embarrassing myself and surely no one would stay at the restaurant longer than 90 minutes.
Well, being weekday mid-day, the restaurant was nearly empty and there was some guy who sat there by himself for over two hours. I ended up taking some of my standards and doing what I could to stretch them out, playing lots of repeats, doing extended improvs over the changes. I turned Herbie Hancock’s “Chameleon” which I normally did as about a 4–5 minute piece following the structure of the Maynard Ferguson version which was the basis for the charts that I originally learned the song from into a 10 minute piece with extended improvs and some bonus repeats. “Night Train” turned into another 10-minute number. And this guy just did not leave. Finally, when I was almost out of material, he got up, dropped a twenty¹ in my tip jar and headed out.
⸻
1. I also got paid by the restaurant, but as I recall, he was the only customer out of the single-digit number of diners in the restaurant who left a tip.
> Most music is highly repetitive, often recycling 2 or 3 short segments (chorus, verse) with minor variations to fill out a whole song. Coltrane is known for his avant-garde composition, and even he repeats (often on a much smaller scale than a pop tune).
I'll note that a pop music current that has, I think, even less repetition than someone like Coltrane was the Progressive Rock genre, especially the originals in the early 1970s. Bands like Yes, Genesis, King Crimson, Pink Floyd, Gentle Giant - they were all producing 10-20 minute songs that used repetition sparingly, instead of relying on a typical verse-chorus-verse structure. Not exclusively, mind, but each had quite a few, typically as centerpieces of their albums.
I'll give them a listen. I've tried to listen to a few things like Marillion and Dream Theater before, but it didn't seem to me that they get anywhere close to something like Yes' Close to the Edge. I'd be happy to find some other band that maybe does, especially if they are even newer.
My entire late 80's high school days were pretty much just listening to Marillion over and over again until my Walkman would wear out the cassette and I'd have to get another copy. Just something about Fish's brooding bitter anger appealed to my angsty teenage mind. Once Fish left though, they quickly moved away from what I was interest in listening to.
These days the one prog album I seem to keep going back to is "Tales From the Lush Attic" by IQ, although "Relayer" by Yes has been getting some good play lately.
I am in the beginning of a personal re-awakening with Marillion. I largely stopped following the band after _Seasons_End_ (despite liking that album). Much like Pink Floyd, I felt that the Whole was greater than the Sum of the parts: without his band to keep him grounded, Fish went into the weeds. Without their jester, the band seemed lessened lyrically.
But: I recently started exploring what they've done in the decades hence, and I am gaining a new appreciation of Hogarth's vocal expression. I find the studio/album version of the song "When I Meet God" to be particularly haunting.
P.S. On Youtube there is a 2-person band named Fleesh that has done spectacular covers of songs like /SFaJT and /Season's End/. Highly recommended.
I keep going back to them occasionally to see what they're up to. I agree, Hogarth has some great vocal skills. "Memory of Water" on "This Strange Engine" is stand out, and much of the album "Brave" is really good. But they still have never really clicked for me. Definitely agree on Fish going into the weeds. I really, really wanted to like his solo stuff, and I do go back to "Vigil" occasionally, but most of his stuff is just rambling and doesn't have the backing band to make it work.
ETA: Oh my goodness, that lady in Fleesh has a beautiful voice. Their version of "Sugar Mice" is stunningly great.
I’ve found that Fish solo is much more satisfying than Fishless Marillion. Steve Hogarth’s vocals seem completely without character to me, especially compared to Fish.
You might like Spock’s Beard. The first 6 albums with Neal Morse have kind of a Yes meets Gentle Giant vibe to them. I like the later stuff as well, but I remember introducing a prog-loving friend to them with a post-Neal show and it was interesting how he managed to pick out the Neal Morse–era songs (and only those) they did as his favorites.
I'm not a professional musician, but I've played and performed in different settings for about 15 years.
Until reading your comment, I didn't realize that many people don't get songs stuck in their head. I often get very long and complex songs stuck in my head for hours to days. Sometimes it's just portions, but sometimes it can be the full song.
I don't quite know how to describe my recollection of music, but it seems to be somewhat akin to eiditic memory, but for sounds. After looking into things, the term eichoic memory seems to come up, but I'm not sure it quite encapsulates what I'm trying to express.
Out of curiousity, do you have aphantasia? I do, and often wonder if the strength of my music recollection comes from a lack of visual recollection in my mind.
Sometimes it takes a few listens to fully memorize a song, but I can revisit music many years later and still have perfect recollection of the piece despite the complexity of the music I tend towards.
For example, my favorite band is Between the Buried and Me (a progressive metal band) and their compositions tend to use a ton of complex and mixed meter, as well as non-repetitive rhythmic patterns. Despite all that, I have perfect recollection of their music—even for songs over 17 minutes long.
Sorry for the long response—your comment just triggered some things I've been thinking about for a while and I wanted to process them and share.
I have aphantasia, and I also remember songs in multiple parts vividly. I won't go so far as to say it's eichoic--I can't, for example, turn around and just play a version of it on piano like some people I know, and I'm fairly sure I'm recalling an approximation in a lot of cases--but it's a detailed enough recollection I can "listen" to songs that way by recalling them. It's very much in contrast to my non-existent ability to recall even a familiar image.
I've considered cause/effect on this before and personally just decided it was probably coincidence combined with my poor visual memory making the other types seem that much more impressive. There's no particular compensatory mechanism I can think of that would explain remembering sound better because I remember vision worse.
Interestingly, I've never been any better at composition (i.e., imagining novel music) because of this. To the extent I've composed, I still pick stuff out a part at a time and figure out how it fits rather than starting with a finished sound in my head to recreate.
I view things like perfect pitch as distinct from a strong musical recollection. With effort, I can transcribe music and eventually learn it, but it's a separate skill in my mind.
The ability to "listen" to songs by recalling them is more what I was referring to when discussing my experiences. I'm with you on the non-existent ability to recall imagery.
I have no idea if there is some type of connection between the two, but it's interesting that at least two of us seem to have similar experiences!
A lot of Baroque music is through-composed. For instance, the Prelude and Fugue forms that J. S. Bach composed some of his music in don't contain outright repetition, or not very much. Themes recur, of course, but in different settings: differently harmonized, in different keys and so on. The two-part form features repetition that may be omitted: while it's common to perform AABB, it can just be AAB, or AB. Sometimes there is a slightly alternative ending in B for when it is followed by A again. The Rondeau form has explicit repetition: AA BA CA ... XA.
Baroque music is often highly repetitive (even fugues are a form of material repetition!) - I would think there was a distinct trend from the 1600s to the mid 1900s where classical music became more and more through composed, until minimalism became suddenly popular.
I think, you cannot have unity in a musical work if there is something new at every turn which does not reappear in any shape or form. In the best music from the Baroque era, there is new material throughout a piece. Many passages introduce motifs that do not make a reappearance. Or make a disguised reappearance just once.
I think, you will be hard-pressed to find two identical bars in a Bach fugue, where all the voices are doing exactly the same thing that was heard before. Even if a recurring theme appears, it's in a different way. Blatant copy and pasting is basically anti-fugue. Fugue-fail.
I'd agree it's rare to have exact repetition of a whole measure across all staves in a fugue, but you can certainly write a fugue using an initial theme and nothing but copy & paste with pitch shifting. Which you couldn't do for, say, a typical solo piano work by Scriabin or Messiaen...
"On the opposite end of the spectrum is Elliott Carter. He intentionally tried to avoid repetition in his works."
Yes, exactly ! That is why Elliott`s work sound like instruments talking to one another, like 4 instruments talking to each other with emotions & expressions each instrument having a specific character.
On the other hand generally we perceive some degree to repeatability as music.
My stepfather is a composer who follows in the path of Ives, one of Carter’s influences. Have been to quite a few concerts by many composers and musicians emulating Ives and Carter and others. As an amateur jazz & classical musician myself, I can understand some of the ideas my stepfather talks about, in the sense of academic concepts. The music in general doesn’t grab me or move me other than in fleeting moments perhaps, the lack of (to me) discernible patterns leaves me feeling like I have nothing to grab onto. I’d love to hear how you (or anyone) enjoy this music, what about it inspires you, what you like about it, does it evoke emotions, how is it similar or different from more mainstream music, etc. For a long time the question in my head has been, am I not enjoying this music because I don’t understand it, or am I just a person who enjoys only certain kinds of music.
> I’d love to hear how you ... enjoy this music ...
I can’t say I do enjoy that music, to be honest. When I was in my late teens and early twenties—as I said, around forty-five years ago—I was attracted by the notion that some 20th century classical composers were breaking new ground, throwing off old-fashioned constraints, being revolutionary, etc. I had studied traditional music theory and learned to play Mozart, Haydn, Beethoven, etc. on the piano, and those more recent composers’ formal experimentations seemed exciting. My youthful rebellious infatuation with their music didn’t last long, and ever since I have listened to more conventional tonal music—not only Western classical but also rock, folk, jazz, reggae, etc.
I did get a bit more out of Elliott Carter when I was young, though, than I did out of other avant garde composers. Amid all the disorder there was something trippy and emotive about his music. These days, for a similar effect, I listen to Bach fugues, which have been mentioned by other commenters here. If anyone is interested, here’s an arrangement of the Art of Fugue that I listened to for the first time the other day and really liked:
> He intentionally tried to avoid repetition in his works. I was into his string quartets for a while about forty-five years ago but haven’t heard them since.
I'd argue that the lack of repetition led to things being less memorable.
> On the opposite end of the spectrum is Elliott Carter. He intentionally tried to avoid repetition in his works.
Aha! So this is what the local classical radio station was playing during its avant garde sets.
Elliott Carter - String Quartet No. 1 is, I find, a deeply deeply irritating work, moreover its advocates are pretentious in the same vein that avant garde jazz muzos are, witness the comment, "Carter always wrote such visceral, intense music for strings. I suspect he really loved the physicality of string instruments." Yeah, right, sure. Call me a philistine, I don't care. Spare me.
I think this is right. A lot of my favorite music finds ways to rotate what's repeating. It finds a balance of keeping the groove while still playing with the listener's expectations. But, to your point, if there isn't enough repetition, there's not enough to grab hold of. Obviously, it depends on what you're trying to accomplish, but for most applications, you want to give the listener engaged with your music.
Beyond, or beneath, song structure, I've noticed that experienced songwriters repeat notes in their melodies more often than people starting out.
mpv skips forward a minute with the up key (sometimes to keyframes by default, but not with flac it seems) and it surprises me a bit how often you can do that and the music doesn't change at all, or only changes slightly :/.
You know that feeling when you wake up in the morning, and the first thought in your head is the sound of the song that's been there for 3 days already? You must have been dreaming the song before you woke up!
The types of error made by the model in the article are exactly the sort of error that I (senior engineer / decade of experience) would make if asked to describe how something works and I didn't want to spend too much time on the details. The difference is, I can usually give a decent estimate of how sure I am of something, and know how to zero in on the truth when it's important.
Maybe a confidence level for a given explanation, along with some sort of "here's where you can go to learn more" would be useful? No idea if language models would be good at that kind of meta reasoning.
There’s already a per-token confidence score given by GPT-3 actually. As for citing actual sources, retrieval architectures like RETRO are becoming popular for this as they can tell you which sources they used for the current prediction and are actually faster thanks to including a fast KNN lookup over the dataset.
> But there's no way to train the transformer-based high-probability-next-word AIs to be superhumanly good at fooling you into doing something, on the grounds of lack of training data
The conversations of all those human scammers would be prefect training data for this. You even know exactly what conversations led to payouts. Assuming you can get all your data in one place, of course.
My context is someone who isn't already falling for the scams. It is true that you can train a model to follow through to those who fall for the scams the scammers already know, which is a fair point. My point is that you're not going to get a superhuman AI out of our current transformer technology that can talk you into believing you're a superintelligent camel from Arcturus IV and if you don't immediately turn over your credit card number, the Star Alligator of the Galactic Core is going to eat your homeworld.
GPT-3 may even gamely try to do exactly that with the correct prompt! But it'll fail. The result won't be cognitively dangerous to anyone with a grip on reality, it'll be risible.
The issue is that when you are trying to understand/modify someone else code it always comes down to confusing implementation details rather than abstract architecture on top of them.
Perhaps "break up your book into chapters" is a better metaphor for microservices. Breaking a chapter into paragraphs makes me think more of OO design or functional decomposition.
It’s breaking up into whole books. Each has is stored, distributed, addressed and built separately. You have to become an expert at making the implied overhead efficient, because it will dominate everything you do.
