AutoTwin-AI: Spatiotemporal ConvLSTM Welding Inspection
In-Situ Predictive Robotic Weld Monitoring & 3D Digital Twin (HackNIMA 2026 Finalist — Presented Sep 26, 2026)
AutoTwin-AI v2: Spatiotemporal ConvLSTM Welding Inspection
SPATIOTEMPORAL 5D TENSOR & NEXT-FRAME PREDICTION
AutoTwin-AI v2 advances beyond static 2D image analysis into a continuous predictive spatiotemporal deep learning framework. Streaming 30 FPS radiometric thermal video is ingested into 5D PyTorch tensors:
The network couples TimeDistributed 2D CNNs (which extract spatial molten weld pool boundary geometries) with ConvLSTM2d recurrent layers (which model thermodynamic fluid flow and non-linear cooling rates across time). The model operates via unsupervised next-frame prediction:
Under nominal steady-state welding, predicted thermal frames match the actual cooling curve, maintaining a near-zero MSE residual. When a transient anomaly occurs—such as a subsurface porosity void, spatter burst, or lack of sidewall fusion—the ConvLSTM prediction diverges sharply, triggering real-time line-trip alerts in under 40ms on NVIDIA TensorRT.
EYE-IN-HAND TCP TELEMETRY & 3D DIGITAL TWIN HUD
The physical sensing rig features a Long-Wave Infrared (LWIR) radiometric thermal camera mounted directly on the robotic manipulator end-effector, locked to the Tool Center Point (TCP). Real-time telemetry is streamed via FastAPI and WebSockets into a React 18 + Three.js 3D inspection dashboard:
Maintains constant focal distance and viewing angle relative to the active arc regardless of 6-DOF robot arm kinematic maneuvers.
Computes active arc duration, localized peak temperature gradients, and spatter ejection rate mapped directly onto the 3D digital twin mesh.
PERFORMANCE BENCHMARKS & EDGE DEPLOYMENT
- Presented live to the international jury panel during the HackNIMA 2026 Final Round on September 26, 2026.
- Overcomes static 2D vision limitations by modeling continuous fluid-thermal weld pool dynamics across time.
- Engineered a sliding-window queue buffering 5D PyTorch tensors for real-time inference on NVIDIA TensorRT.
- Integrated telemetry pipelines computing active arc duration and spatter density index (S_dot) streamed via FastAPI to a 3D WebGL dashboard.