WEBVTT

00:00:01.290 --> 00:00:02.212
Hi, I'm Mocha!

00:00:02.212 --> 00:00:05.899
SAM 3 tracks every object in a video, and its cost grew

00:00:05.899 --> 00:00:07.435
with each one it tracked.

00:00:07.435 --> 00:00:09.586
Here's how SAM 3.1 shares the work.

00:00:11.142 --> 00:00:14.457
SAM 3 ran a memory path for every tracked object.

00:00:14.457 --> 00:00:18.435
Object Multiplex packs up to sixteen objects into a bucket, and runs

00:00:18.435 --> 00:00:19.761
it once per bucket.

00:00:21.342 --> 00:00:23.796
The frame is encoded once, as before.

00:00:23.796 --> 00:00:28.004
Then up to sixteen objects' masks are packed into one bucket, and

00:00:28.004 --> 00:00:30.458
encoded together into one shared spatial memory.

00:00:30.878 --> 00:00:34.352
What keeps the objects apart is a small embedding for each one,

00:00:34.352 --> 00:00:36.958
stored in the memory bank beside the shared features.

00:00:37.378 --> 00:00:41.477
Memory attention and the mask decoder run once for the whole bucket.

00:00:41.477 --> 00:00:44.893
Each object's embedding is simply added to its mask queries.

00:00:45.313 --> 00:00:47.946
Demux hands every object its own mask.

00:00:47.946 --> 00:00:53.212
Sharing works because sixteen objects in a frame mostly sit in sixteen different places.

00:00:54.692 --> 00:00:57.883
Fit a straight line to Meta's own eight measured points, and frame

00:00:57.883 --> 00:00:58.947
time looks like this.

00:00:59.367 --> 00:01:03.993
Twenty-nine milliseconds is the floor: the backbone, which both versions always shared.

00:01:04.413 --> 00:01:07.152
Each extra object adds under half a millisecond.

00:01:07.152 --> 00:01:09.890
SAM 3 paid four point six four five.

00:01:11.592 --> 00:01:14.195
So the speedup depends on where you stand.

00:01:14.195 --> 00:01:16.798
At one object it's about seven percent slower.

00:01:16.798 --> 00:01:20.051
At a hundred and twenty-eight, five point two times faster.

00:01:21.642 --> 00:01:25.502
The speedup is a slope, not a constant: the same backbone floor,

00:01:25.502 --> 00:01:28.076
with a far cheaper cost per extra object.

00:01:30.192 --> 00:01:34.506
To recap: sixteen objects share one memory pass, small embeddings keep them

00:01:34.506 --> 00:01:37.381
apart, and the gain grows with every object.

00:01:37.801 --> 00:01:39.875
Every source is in the full article.

00:01:39.875 --> 00:01:40.468
I'm Mocha.

00:01:40.468 --> 00:01:40.765
Bye!
