Human-guided cross-domain synthesis: generating virtual robotic arm imagery and videos
Currently, a multitude of interaction methods between humans and robotic arms have emerged, among which one effective strategy is to enable robotic arms to imitate human arm movements, thereby achieving intuitive operation. With technological advancements, robotic arms are now capable of learning...
محفوظ في:
المؤلف الرئيسي: | |
---|---|
مؤلفون آخرون: | |
التنسيق: | Thesis-Master by Coursework |
اللغة: | English |
منشور في: |
Nanyang Technological University
2024
|
الموضوعات: | |
الوصول للمادة أونلاين: | https://hdl.handle.net/10356/173713 |
الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
المؤسسة: | Nanyang Technological University |
اللغة: | English |
الملخص: | Currently, a multitude of interaction methods between humans and robotic arms
have emerged, among which one effective strategy is to enable robotic arms
to imitate human arm movements, thereby achieving intuitive operation. With
technological advancements, robotic arms are now capable of learning and imitating
actions by watching their videos or images. This dissertation proposes
a method using cross-domain conversion and image generation technology to
transform videos of human arm movements into robotic arm action videos. This
method provides real robotic arms with opportunities to learn and imitate, further
enabling direct interaction by mimicking human arm movements. By processing
videos into frames and utilizing adversarial generative networks and contrastive
learning frameworks, the mutual information between input and output
domain image patches is maximized, effectively achieving cross-domain conversion.
Moreover, to enhance the model’s generalization capabilities, techniques
such as image masking and human skeleton keypoints detection have been introduced.
This not only broadens the scope of the model’s application but also
provides insights for tasks involving cross-domain conversion and opens up additional
possibilities for the learning of robotic arms. |
---|