We can't eat our cake and have it to, so we have to decide: Monitoring the internet to catch the bad guys while opening our country up to more extensive sabotage, or promoting privacy (even when it can be misused by some criminals) and protecting our lives from foreign interference. Obviously it isn't a dichotomy like this, but it makes the point that we either have security for everyone, or no one.

They're monitoring their own servers. The data is there either way.

@murgatroid99 They put monitoring infrastructure in place. That is where the taps go.

It's not like it's some specific physical place

Without that (or better yet, with e2e), it's a lot more difficult. You ever tried scanning for complex regex on a pwnt low-end Cisco router? It's nearly impossible. They're fast because of hardware NAT, but their CPUs have only a paucity of cycles to spare. An ML-based image monitoring server exporting heavily-processed metadata through said router full of all the preprocessed salient data anyone could want in a direct stream to the servers that run the policies, on the other hand...

(Actually it is, at least for large companies with a complex network topology like Google)

I'm pretty sure they're checking the data while it's at rest.

It's not a rest when it's going between their servers to do the checking.

It's not all done on a single server (i.e. the server storing the data is not the server running the ML algorithms which is not the server handling the policies which is not the server that manages routing). It's the leaf nodes (the "dumb" devices sitting on the edge of all of this) that are juicy targets, not the servers themselves.

But probably none of those are on any specific server. And data is encrypted going between data centers. And if this specific monitoring job was removed, it wouldn't meaningfully reduce how frequently this data moves between data centers.

Between DCs, sure, but not necessarily between servers within one DC. They won't dedicate an entire datacenter to one purpose like that (except perhaps simple storage). They use virtualization to ensure low-latency and failover capabilities. Cattle, not pets...

I can say with absolute confidence, as a hacker, that it is infinitely easier to monitor something that already has monitoring infrastructure in place than it is to set up your own infrastructure, undetected. This isn't hypothetical either and this kind of thing has happened (and continues to happen on a regular basis). If they switched to e2e, this would be next to impossible, at least on a dragnet scale (obviously targeted monitoring would still be feasible by tapping the endpoints).

I don't really understand what threat model you're proposing here

Think from an attacker's perspective: You don't have all the cycles you want. You can only do some very limited processing before you get caught. You also have copious traffic that you want to monitor. Do you a) try to DPI all that traffic yourself and get caught and lose your shell, or b) take advantage of the data that has already been deeply processed by one of Google's NN-accelerated servers where all you need to do is run some really, really trivial regex to get what you want?

Because all that NN-accelerated processing is happening on a different server, so the traffic will be transmitted internally, ripe for the picking by even the lowest-end dumb router or hypervisor with no more than a spare cycle here and there.

Yeah, I'm sure that the isCASM boolean result will be of great use for hackers who want to know if a picture is CASM or not.

That's not how it works.

It doesn't simply return true or false. It offloads the processing which does far more complex analysis and allows the server which sent the query to parse the results. That makes it easier to upgrade as you no longer have to work with unprocessed images and can instead write a ruleset for processed data.

(deleted this para because it was a terrible and confusing analogy)

In general, if you're writing complex rulesets, it's far easier to write a rule-agnostic processor and then write the rules on the processed data than it is to change the configuration on the processor itself. For example, Linux LSMs work that way (the hooks are strewn throughout the kernel so the rules only have to deal with LSM hooks rather than all of the kernel code).

@murgatroid99 So, as a flawed analogy, it's like the preprocessor returns brightness, color, contrast, size, etc. rather than "is_blacklisted:true" or "is_blacklisted:false" (since it's actually ML, it's a much more complex series of weights and allows you to determine a lot of information about an image without seeing it, but that's the general idea).

So while it's absolutely true that there's no point in sniffing the "is_blacklisted" field, sniffing ML weights is far more useful...

@murgatroid99 So regardless of where you stand morally in regards to the surveillance-vs-privacy debate, it is absolutely true that a surveillance-supported infrastructure makes it much easier for the "bad guys" to do the same, and those bad guys are anyone from the Russian government trying to rig an election to industrial espionage actors who want to sell your medical info to a crazy hacker with the skills of a TAO member but the mental maturity of a 14 year old script kiddie.

Yeah, it doesn't look like that's how it works. I don't see exactly how the internals of the service work, but the API is more opaque than what you're describing.

Why do you say that?

I'm talking about the intra-DC infrastructure, not the public (or even private) API.

Also remember that it will go through layers of abstraction until it eventually does reach a point where it hits an is_illegal check. This is how it works in all organizations with complex network infrastructures. Google is no exception.

It's only the relatively small operations where you can have 100% unprocessed input come into a single server and 100% processed metadata with all the booleans required for making policy decisions come out of it.

Really, we're both just speculating here. Neither of us knows how exactly this component works, but it's not really implausible for the steps you're describing to happen within a single process.

I'm not speculating. This is how it works.

Oh, no not a single process. I was simplifying.

Really, you've read the code?

No, but I know how companies like this operate. It's for the same reason that I know that they use custom Linux kernels without reading the patches. I also know they use remote attestation despite not having even looked to see if they have TPMs installed.

I'm pretty sure that's just actual public knowledge. The implementation details of specific internal services are not

Sure, and I don't know the details. I don't know how many ML weights they use or if they use 3 or 30 servers to do that. I don't know if they communicate using GRE or IPIP or whatever. But I do know that they do not have an opaque server that takes in a JPEG file and spits out a boolean "is_cp".

How much experience do you have running these kinds of services?

Oh, none

How about in lateral movement within an organization after hacking it or monitoring salient traffic from routing hardware?

Anyway, we've been having this conversation for over two hours, and I have other things to do. So, I'm done.

OK. Have a good time. The last thing I'll say is that I do have experience in the relevant fields, which is why I know what infosec risks Google is taking by doing this. I'm not making this up or throwing out wild speculations.

So the post pandemic version of 9/11 is 9/3?

Wow! So many calls on twitter to "shoot him down!!!"

i think that's the official response during an airplane hijack.....what's that, he wants to hit a Walmart? how many people in it. it's empty!? oh well then tell the pilot to go fuck himself and blow the place up

I watched a documentary called 9/11The Final Minutes, where they had the time to get fighters up in the air to shoot'em down; but nobody was willing to make the decision.

Instead, they kept arguing with each other and not acknowledging the real danger.

They actually heard the hijacker's announcement to the passengers.

Sounds like he safely landed and is in custody.

thedailybeast.com/…