Pixels become a circuit graph.

One pipeline, different kinds of evidence

The system is designed to infer as little as possible. Some information is already known: the board’s geometry, the available parts, and the exercise’s intended circuit. Some information can be measured: for example, which LED responds when the Arduino drives a pin. The remaining uncertainty—especially component and wire endpoint locations—comes from the image.

1. Capture and normalization — implemented

The student photographs the circuit. The normalizer finds the breadboard, corrects its perspective, and maps the known hole grid into a consistent top-down coordinate system.

The inherited implementation in breadboard/rectify.py is based on Hartley Blakey’s breadboard-normalizer. It uses a coarse document-corner estimate, hole-grid detection, alignment, and the board’s rail stripes to disambiguate orientation. It can refuse to use a poor alignment rather than silently assign the wrong hole numbers.

This is a working subsystem, not a guarantee that every photograph will normalize successfully. The previous team identified failures on a subset of challenging images.

2. Visual perception — our current engineering focus

Once the grid is normalized, a perception model must answer:

• Which components appear in the image?
• Where do their leads or wire ends enter the breadboard?
• Which endpoints belong to the same physical object?
• Which predictions are confident enough to use?

The output must be sufficient to describe the student’s actual circuit as structured data. An earlier model can detect resistors and two keypoints, but the whole circuit—including LEDs, sensors, and overlapping wires—remains unsolved.

Explore the computer-vision problem in detail.

3. Structured circuit data — interface implemented

The system’s breadboard/netlist.py defines the data model. A component can be described by type and connection-hole coordinates; an example fragment might look like this:

{
  "id": "W_blue",
  "type": "wire",
  "pins": {
    "1": { "hole": "a54" },
    "2": { "hole": "a33" }
  }
}

This is an illustrative component fragment, not the full expected JSON file. The existing example builds and Netlist implementation are the source of truth for the exact schema.

4. Circuit checking — implemented

The checker represents the electrical circuit as a graph: electrical nodes connected by components. It can compare a proposed build to the required lab circuit and identify certain wrong or missing connections, reversed components, or uncertain observations.

A significant design choice is that different physical layouts of the same electrical circuit are acceptable. The checker operates on electrical topology rather than demanding a single visual arrangement.

5. Arduino-assisted diagnosis — existing prototype

Photographs cannot prove all electrical behaviors. The previous team implemented USB-based Arduino interrogation and an LED “blink and watch” routine that uses camera observations to determine which LED responds to a given output pin.

These tools have been demonstrated on real hardware by the previous developers. Our team’s end-to-end classroom hardware setup has not yet been verified, and Chrome deployment remains future work.

What is currently demonstrated?

Review current development status · Browse the actual implementation