A hiking app names plants live through the phone camera. Trails often have no signal, so the model has to run on the phone. The team wants the largest model the phone can handle, since within this family bigger has meant more accurate.
Time. The camera delivers frames per second. Every frame passes through three stages, one after another:
| stage | time per frame |
|---|---|
| preparing the image | ms |
| running the model | depends on the model |
| drawing the label on screen | ms |
A frame that is not finished before the next one arrives makes the overlay stutter. The team accepts at most 1 late frame in 100.
On this phone, a test model with million parameters (8-bit weights) was timed over thousands of frames:
| measurement | model time |
|---|---|
| half of all frames finish within | ms |
| 99 frames in 100 finish within | ms |
Model time grows in proportion to the number of parameters, for both measurements.
Memory. The phone shuts down any app that uses more than MB. The rest of the app (interface, camera buffers) already takes MB. While it runs, the model needs MB of working memory whatever its size, plus its weights, stored at 8 bits each. Take MB bytes.
What is the largest model, in millions of parameters, that meets both requirements? Round to 1 decimal place.