1–6 Jun 2025
Taipei International Convention Center (TICC)
Asia/Taipei timezone

LLM and CV for lab assistance agents

THPM108
5 Jun 2025, 15:30
2h
Exhibiton Hall A _Magpie (TWTC)

Exhibiton Hall A _Magpie

TWTC

Poster Presentation MC6.D13 Machine Learning Thursday Poster Session

Speaker

Karel Peetermans (Deutsches Elektronen-Synchrotron DESY)

Description

Recent advancements in large language models (LLMs) have enabled them to solve complex tasks using natural language prompts. Similarly, computer vision (CV) continues to enhance machine capabilities in identifying and interpreting objects and scenes. This work explores the integration of LLMs and CV to improve laboratory operations, focusing on automating tasks like data collection and process monitoring.

We present a system designed to streamline laboratory workflows by combining a locally deployed LLM with a YOLO-based object detection model. The system assists in locating and identifying equipment, tracking positions, accessing parameter settings, and recording experimental data in real time. This integration enhances task automation, accessibility, and precision in laboratory environments.

In conclusion, merging LLMs and CV techniques represents a significant step toward creating intelligent and efficient laboratory assistants.

Region represented Europe
Paper preparation format LaTeX

Author

Bianca Veglia (Deutsches Elektronen-Synchrotron DESY)

Co-authors

Frank Mayet (Deutsches Elektronen-Synchrotron DESY) Ilya Agapov (Deutsches Elektronen-Synchrotron DESY)

Presentation materials

There are no materials yet.