Meeting notes — 2026-09-23

Updated: 2026-09-30

Recorded discussion, 15:30. Topic: dynamic throwing and whether tactile sensing is load-bearing. Transcribed from audio; several technical terms were garbled by speech recognition and are corrected inline.

Follow-up session: 2026-09-30.

#Data pipelines considered

Discussion opened on an objection that the proposed data collection was "a very manual effort", with the reply that this is how the field currently does it, "which is why nobody ever wants to". Three candidates:

Sim-as-exploration, real-world adaptation. Use simulation only to generate exploratory/play behaviour, then run those randomized behaviours on the real robot. Per-rollout success does not matter as long as some succeed. Record tactile alongside the plays and train on that real data. Accepted as a workable starting point — "the details are okay, we can figure it out along the way".
Real-only play data, learned world model, MPC. From a recent paper: collect a large amount of real play data and learn to predict the bottle trajectory, giving a world model. A play policy proposes throws; roll out hundreds of proposals; solve an MPC problem over the learned model to plan a throw that lands the bottle upright. No simulation.
Domain randomization in simulation. Randomize water level, centre of mass, and bottle weight. Judged doable, but later questioned as the wrong framing (see below).

Some subset of these has to be done regardless: if the group invents a different method, these become the natural baselines.

#Is tactile central?

Concern raised that tactile "is not the central focus" — clearly helpful, but not load-bearing. The setting that resolved it:

You are handed a water bottle.
No camera to look at it, and the water level is not disclosed.
You are told where to throw; it must land in a target box within some error.

First reaction was that this is impossible, on the grounds that real-world play policies are tuned to the one specific bottle you practise with. After clarifying that the water level is genuinely unknown at test time, the group converged: in that setting tactile becomes essential.

The crux: with no observation that disambiguates weight, a policy can only learn the mean throw — it cannot adapt. Being told where to throw while not knowing the object is exactly what forces an adaptive controller, which requires on-the-fly system identification.

#Preferred approach — latent sys ID over domain randomization

Pushback on solving this "in this kind of domain randomization type way". Preferred alternative: use tactile for fast system/physical identification of the whole system on the fly, which is roughly what humans do given many attempts. Acknowledged that a colleague (Roshi) might consider this less novel.

Key refinement: do not expose explicit parameters for centre of mass, inertia, and friction. Keep those properties latent and identify them on the fly — this covers a wider range of situations than a parametric description.

SwingBot (2020) — in-hand tactile exploration to infer physical properties, including centre-of-mass shift, for dynamic swing-up manipulation. Also likened to Roshi's paper-airplane work. Differences: SwingBot's representation is parametric where we want latent, and it demonstrates a single task. Doing it across more tasks, faster, is the advance.

#Scope — "toss anything"

Proposal to avoid a single-task demo: pick any object from a bin and toss it into another bin. Whether this is the same problem was debated:

One view: if you can throw a water bottle at unknown weights, blind, you can more or less handle different objects already — it is a framing question.
Counter, accepted: the requirements differ. The bottle task fixes shape and varies weight, and is fine-grained and dynamic. "Toss anything" is geometry generalization plus weight/attribute variation.
Ideal outcome: show both with one general latent-sys-ID-plus-tactile approach that assumes no explicit physical parameters.

Is tactile necessary for generic tossing? Left unresolved.

A parallel-jaw gripper could plausibly toss many objects, so tactile is not clearly required. A long rope was raised as needing a dexterous hand to wrap around — but that argues for dexterity, not tactile.
Whether RGB state estimation is permitted is the deciding variable. If you may look, tactile's necessity weakens for generic tossing. For the unknown-water bottle it is still needed even with a camera, since vision cannot read fill state reliably.
The one concrete tactile argument offered was estimating object weight.

#Decisions

Get started; the setting is interesting enough.
Explicitly no conclusion on method choice.
Next step: train the policy first, with a bottle at different water levels. Random-object tossing deferred behind the bottle task.
Follow-up invited on what each person specifically wants to pursue.