Robotics research concept · V15

NTRAM

Neurally Trained Robot Arm Model · simulation-trained vision and movement

Scene 0
About NTRAM · how it works

NTRAM (Neurally Trained Robot Arm Model) explores whether a simulated robot can learn to recognise coloured cubes and choose safe movements, then perform the same task using a single RGB camera and learned neural networks without virtual touch sensors.

  1. Simulation teacher. During training, programmed colour recognition labels camera pixels with the correct basket. Virtual touch supplies safe and collision examples.
  2. Pixel vision. A 384 × 384 RGB camera supplies the images. A learned radial-basis classifier associates pixel colours with basket destinations; it receives no depth image.
  3. Movement learning. Two neural networks learn collision risk and the success of candidate grasp-and-place actions using joint positions and features derived from the camera. Both safe movements and collisions are retained as training evidence, even when a challenge fails.
  4. Crash map. The 3D graph displays green (safe / low risk), yellow (uncertain / mixed) and red (collision / high risk). Large dots show observed evidence; small dots are neural predictions. It plots base yaw, shoulder and elbow angles, with a selectable tool-pitch slice.
  5. Independent test. Learned vision selects the basket, and learned motion scores rank candidate routes. Teacher colour lookup, touch feedback, exploration and weight updates are disabled. A physical collision ends the test.
  6. Sorting task. Sort every cube into its matching fixed basket without upsetting the stacks. Baskets fill in layers of four. Training uses up to 1,000 attempts or five successful sorts per layout, with rotated viewpoints. Timeouts are unscored; tests earn +1 or −1.

Prototype limits: Camera calibration, cube geometry extraction, fixed basket positions, inverse kinematics and candidate trajectories are programmed. The physics is approximate; the network is not an end-to-end motor controller. No real-robot safety or transfer performance has been established.

Training runs at maximum speed with five-second snapshots. Testing starts at slow speed. Simple training uses 3–6 cubes, Mixed and Tall use 6–12, and new tests use 6–12. Learning is saved locally when supported.

Ready
0Training successes (+1 each)
0/0Cubes correctly sorted
0/1000Layout attempts
0Safe observations
0Collisions learned
—Test net score

0 right · 0 wrong

More controls & statistics
0Training layouts
—Training deliveries (+1; exploration unscored)

0 complete · 0 failed probes

0/5Complete runs in layout
0Touch samples this run
0°Robot heading
0Fallen / disturbed cubes this run

01 / Sorting world

Drag to orbit · observer view
PausedReadyLive view

02 / Learned 3D crash risk

Safe / low riskUncertain / mixedCollision / high riskSmall dots: predicted · large dots: observed

03 / Robot RGB camera

384 × 384 RGB pixels · no depth buffer
Awaiting image.

04 / Learning and tests

Training completed sorts · frozen test net score