Yeji Song

Research Scientist

ldynx.github.io/

https://github.com/ldynx

ldynx@snu.ac.kr

I am a Ph.D student at Seoul National University, under the supervision of Prof. Nojun Kwak. My primary focus is on video & image generation, aiming to push the boundaries of their applications in real-world scenarios. Specifically, developing generative models that provide more diverse experiences to users is my central goal. My research interests also include a broader computer vision area, with experience spanning diffusion, video rendering, segmentation, and 3D object detection.

Education

Seoul National University
Ph. D in Computer Science, advised by Prof. Nojun Kwak.
Supported by LG Electronics Scholarship 2023-2025
Spring 2020 - Fall 2025 (Expected)
Seoul National University
B.S. in Statistics and Psychology (double-major).
Spring 2013 - Spring 2020

Employment

NAVER Webtoon
Applied AI Intern on Webtoon AI team. Worked on Semantic Segmentation for Webtoon Platform.
Jan 2023 - July 2023
Korea

Publications

Video Generation & Editing

SAVE: Protagonist Diversification with Structure Agnostic Video Editing
Y. Song, W. Shin, J. Lee, J. Kim, N. Kwak
We adopt motion personalization in video editing tasks, isolating the motion from a single source video and subsequently modifying the protagonist accordingly.
ECCV, 2024
Towards Efficient Neural Scene Graphs by Learning Consistency Fields
Y. Song, C. Kong, S. Lee, N. Kwak * , J. Lee *
In video scene rendering, we reformulate neural radiance fields to additionally consider consistency fields, enabling more efficient and controllable scene manipulation.
BMVC, 2022

Image Customization

Harmonizing Visual and Textual Embeddings for Zero-Shot Text-to-Image Customization
Y. Song, J. Kim * , W. Park * , W. Shin, W. Rhee, N. Kwak
There is a conflict among contextual embeddings in zero-shot T2I customization when varying the subject's pose. We resolve it by orthogonalization and attention swap.
arXiv, 2024

Text Detection

Exploring Causal Mechanisms for Machine Text Detection Methods
K. Yoo, W. Ahn, Y. Song, N. Kwak
With the increasing importance of discriminating machine-text from human text, we show the existence of backdoor path that confounds the relationships between text and its detection score.
NAACL Workshop, 2024

Pruning

Finding Efficient Pruned Network via Refined Gradients for Pruned Weights
J. Kim, J. Yoo * , Y. Song * , K. Yoo, N. Kwak
We advance dynamic pruning by employing refined gradients to update the pruned weights, enhancing both training stability and the model performance.
ACM MM, 2023

3D Object Detection

Md3d: Mixture-density-based 3d object detection in point clouds
J. Choi, Y. Song, Y. Kim, J. Yoo, N. Kwak
We utilize the Gaussian Mixture Model (GMM) in the 3D object detection task to predict the distribution of 3D bounding boxes, eliminating the need for laborious, hand-crafted anchor design.
IEEE Access, 2022
Part-aware data augmentation for 3d object detection in point cloud
J. Choi, Y. Song, N. Kwak
We delve into data augmentation in 3D object detection, leveraging sophisticated and rich structural information present in 3D labels.
IROS, 2021

Projects

Video Generation & Editing

Video outpainting for Media Art
Easywith Co., Ltd.
• Tackled the physical limitations in the Media Art Exhibition derived from exhibition space which has a different size and ratio from artwork.
• Adopting the diffusion-based video method, the artwork has been outpainted to match the size and ratio of the exhibition space.
Nov 2023 - Feb 2024

Vision Application

Webtoon translator machine
Naver Webtoon Corp.
• Rearranging balloons and typesets is necessary during webtoon translation, and proper segmentation for these elements becomes essential.
• Took charge of the entire development process of the webtoon segmentation model, from enhancing data augmentation schemes to building a demo page.
Jan 2023 - March 2023
Development of object detector for Fashion AI Chatbot
Ministry of Science, and Information and Communication Technologies (ICT)
• Participated in developing an AI chatbot that facilitates multimodal conversations about fashion, engaging in both text and images.
• Contributed to implementing an object detector capable of analyzing the user-provided images, capturing clothing items, and extracting critical features.
Jan 2022 - Dec 2022
Unmanned outdoor security robot
Samsung Electronics Co., Ltd.
• Participated in a project aimed at creating an autonomous outdoor security robot capable of independent navigation and object detection.
• Enhanced the robot's detection capabilities by leveraging official benchmark datasets and a limited amount of real-world data.
June 2021 - Dec 2021