image of breaking badness
Breaking Badness
Breaking Badness

106. Log4Shell Shock


Here are a few highlights from each article we discussed:

Beware of Threat Actors Bearing Gifs

  • The NSO group is an Israeli technology company primarily involved in developing mobile surveillance rootkits and selling access to them to government and law enforcement customers.
  • The NSO has been all over the news recently as their Pegasus kit has been caught in action by the folks at Citizen Lab on the phones of journalists for the NY Times, Al Jazeera, etc. They’ve also been accused of deploying their Pegasus rootkit onto the phones of US diplomats, French government officials (we covered this over the summer) and a whole host of other targets that violate their own guidelines around the usage of their technology. They’re facing a number of lawsuits, even from Apple themselves.
  • There’s been a lot of work done on the research side for Pegasus lately. The google Project Zero team in particular has done a deep dive into messaging vulnerabilities and iMessage presents a lot of surface area, particularly around image parsing. In the exploit detailed here the problem starts with gif files getting automatically parsed outside of the imessage “sandbox” dubbed BLASTDOOR, implemented earlier this year in iOS.
  • In this case gifs are just the container for a file compressed with the JBIG2 format… an extreme compression format developed for PDFs in 2000. They were able to produce an overflow and gain access to protected memory.
  • In this case the “extreme compression” uses pattern matching and substitution to drastically shrink the size of a text document. It also includes some basic logic flow operators for “restoring” the document properly. The NSO team used the overflow to place enough of their own logic operators into protected memory (70 thousand) to create a virtualized computer architecture…They can make your iPhone dream up a virtual machine that they have access to, in a spot where it should not exist.
  • In practice we’re starting to shed some light on how Pegasus was able to land onto the phones of its targets and we’ve gotten a glimpse into how it may have avoided detection for so long. The Google Project Zero folks have an upcoming part 2 that will get into more detail, but what we’ve learned here is that the NSO team built an incredibly complex bit of software to accomplish this.
  • Apple released a patch in September that helps to plug the GIF parsing issue that allowed NSO to operate outside of the iMessage sandbox.

Walking on Log4Shell Vulnerabilities

  • On December 9, it was revealed that there is a very, very significant vulnerability in an Apache logging library called “log4j.” To put it in perspective, people are calling this vulnerability, in the way that all vulnerabilities have clever names these days, “log4Shell.” And that tells us what we need to know about the nature of the vuln, really, doesn’t it? Pass in some parameters, get a shell. I’ll give more details on that a little later, but just to set the scene here, the log4j library is used in the vast majority of Java applications, and Java applications aren’t exactly obscure. So part of what we’re going to talk about here is the wide scope of this vulnerability.
  • And then the other reason that if you’re listening to Breaking Badness there’s about a 100% chance that you’ve heard of this, is that there’s active exploit activity in the wild. Now, having said that, it’s worth mentioning that from the point of view of a lot of the scanning that was seen in the days right after the exploit was made known, did look like it was benign—it was vulnerability scanning that did not appear to be of malicious origin. But don’t be too comforted by that. There’s plenty of exploitation going around; CISA announced a couple of days after the exploit was announced that they’re seeing a big ramp-up in known threat actors getting in on it; we can only assume that things haven’t tailed off. Jen Easterly, the head of CISA, whose first rodeo this decidedly ain’t, said this might be the most serious vuln she’s seen in her entire career.
  • It’s going to be easier most likely to identify organizations that do NOT have log4j running somewhere. Hundreds of millions of devices are affected, and all of the big players have it running somewhere. Your SaaS vendors, whether from a B2B or a B2C standpoint, are (or at least until very recently were) running log4j somewhere (probably multiple “somewheres.”).
  • You know, I really can’t put why log4j is so ubiquitous any more succinctly than the one and only Randall Munroe, of xkcd fame, did—long before this vulnerability was discovered. I wouldn’t normally describe a drawing in a podcast, but hear me out here; this one’s worth it. It says essentially that the entire Internet depends on “a project that some random person in Nebraska has been thanklessly maintaining since 2003” and it shows your generic block diagram of a whole bunch of blocks and layers that represent modern digital infrastructure, where in this case log4j is this one little tiny block that balances the whole stack and keeps it from collapsing.
  • Apache is one of the most common open-source server packages, and log4j is in Apache. By the way, Kelsey, I saw this one and thought you might appreciate it: what’s a cure for apache log4j? A patchy log4j. You’re welcome.
  • I’m not sure precisely *how* log4j was discovered, i.e. what exact research or hacking was in progress to probe this specific library and with what intentions, but what we do know is that it was privately disclosed to Apache by Chen Zhaojun of the Alibaba Cloud Security Team back on November 24, but publicly on December 9 by a tweet (since deleted) from an account called P0rz9, with a link to a GitHub repo with an exploit PoC.
  • Because log4j is (wait for it) a *logging* library, the way that you can get your shell is by getting a particular string to be logged. Now if you think about how logging works, there are myriad ways to get a particular event to be logged. It can be passed into web servers, it can be embedded in an email or it can even be sent in the username field of a login page. And as for how it’s being used, we know that some of the exploits are things like installing cryptominers, stealing credentials, and basically gain unauthorized access to stage later attacks. Certainly we’ll hear about this as an initial vector in ransomware events, espionage, and lots of other malfeasance.
  • A domain bloom is a pattern of registrations centering on a particular word (really any string of characters, but we are specifically noting these as they pertain to words in dictionaries. Now, in this case, it’s sort of retroactive definition, since “log4j” was not a word in any dictionary sense, but it is now, so this is a bloom pattern). Anyway, what you see in a classic bloom is that the rate at which that word or string appears in domain registrations has a baseline average over time, and then you see a sudden rise in registrations that have that string in the domain names, and the elevated rate of registrations is sustained for multiple days before tailing off to either the original baseline (for words that were common already, like “iphone”) or a new baseline (for words that enter common usage, like “Covid” or, now “log4j.” So this bloom has been going on since the disclosure on December 9, appeared (so far) to peak on the 14th, and has been gradually decreasing (with a couple of bumps along the way) since then. Now – we’re not talking about big absolute numbers. The record day was just 19 domain registrations with log4j in the names. This is a tiny bloom compared to what we saw with Covid, but it’s definitely following the pattern.
  • You need to identify your assets that are running this library, and patch them. Very interestingly, there’s a “vaccine” approach for this, which is not really what’s formally being recommended but which does seem to work, which is that you send a different string to be logged by that library, which causes it to be turned off. This was released by Cybereason, and it’s gotten mixed reviews, so really your best bet is to patch to 2.17.0 (which as of the day we’re recording here is the latest, and recommended, version). There was an initial patch which, itself, has issues, but for now at least 2.17.0 seems to be good.
  • Vulnerability and asset management tools are getting their day in the sun here, without a doubt.

Two Truths and a Lie

Introducing our newest segment on Breaking Badness. We are going to play a game you are all likely familiar with called two truths and a lie, with a fun twist. Each week, one us with come prepared with three article titles, two of which are real, and one is, you guessed it, A LIE.

You’ll have to tune in to find out!

Current Scoreboard

Breaking Badness Two Truths and a Lie


This Week’s Hoodie/Goodie Scale

Beware of Threat Actors Bearing Gifs

[Taylor]: 1.5/10 Hoodies
[Tim]: 5/10 Hoodies

Walking on Log4Shell Vulnerabilities

[Taylor]: 10/10 Hoodies
[Tim]: 11/10 Hoodies


That’s about all we have for this week, you can find us on Twitter @domaintools, all of the articles mentioned in our podcast will always be included on our podcast recap. Catch us Wednesdays at 9 AM Pacific time when we publish our next podcast and blog.

*A special thanks to John Roderick for our incredible podcast music!