Incognition logo
Home
Learn
Practice
Leaderboard
0
VisualizationsApplied Machine Learning

Batch vs Real-Time Scoring

Slide how many rows the model scores at once, for a Monday offer list or a card payment. Bigger batches score more per second, but the first answer comes later.

first rowa row arrives every 25 ms
first row's answer: 22 ms
0 s0.5 s1 s1.5 s
gathering rowsone run: 20 + 2 × 1 = 22 ms
batch size1 row
time per row22.0 ms
most rows it can score per second45
first row's answer after22 ms

One row per run: at most 45 rows a second. Nobody is waiting for these answers, so there's no need to score rows one at a time.

Customize