Build
CODE
Develop
©2023
CA, USA
I am a graduate student researcher in FHL Vive Center, leading the OpenARK team in EECS Dept @UC Berkeley, working closely with the director Dr. Allen Y. Yang. My research interests focus on 3D Vision, Human-AI Interaction and Autonomous Robots with a strong emphasis on their integration into Architectural Construction and Urban Accessibility. Prior to this, I gained experience in the AEC industry and obtained my Bachelor of Architecture from Zhejiang University, leading the robotic fabrication lab in Civil-Eng. Dept.
🎉Recent Highlights
10/2023. I will be giving a 2-hour online talk at Xi'an University of Architecture and Technology. The topic is "From Robotic Fabrication to 3D Scene Understanding."
10/2023. We are releasing DTTD v2! 🎉 A 3D object tracking dataset and a transformer-based 6DoF object pose estimation network that is robust to sensor depth noise.
08/2023. Our paper "MARL: Multi-scale Archetype Representation Learning for Urban Building Energy Modeling" is accepted at ICCV workshop 2023, CVAAD.
01/2023. We won the MIT RealityHACK 2023 in Spatial Audio Track! 🎉
Research Journey
3D Vision
+ MORE
📃Robust Digital-Twin Localization via An RGBD-based Transformer Network and A Comprehensive Evaluation on a Mobile Dataset
AI FABZixun Huang*, Keling Yao*, Seth Z. Zhao*, Chuanyu Pan*, Tianjian Xu, Weiyu Feng, supervised by Dr. Allen Y. Yang
[github] [arxiv] [bibtex] [dataset] in submission to TPAMI
Are current pose estimation methods robust enough to ignore the distortion and interpolation noise in widely-adopted IPhone's LiDAR measurements? 🤔️Our DTTD-Net introduced Fourier-transform enhanced MLP and fusion-robustifying Transformer into 3D Object Tracking Tasks.
Robotic Fabrication
+ MORE
Jieyuan Shrine: Advancing Modular Architecture Construction with a Robotic Carbon-Fiber Weaving System
AI FABKuanting Lai*, Zhe Liang*, Zixun Huang*, Peiyi Huang, Yuhong Ha
[video] [animation] [poster] featured in: [archdaily, domus, gooood]
🎉Our work has been featured in the most prestigious architectural media.🎉
🎉We achieved China’s first all carbon-fiber structure designed architecture. Density of the structure is controlled at 18KG per cubic meter and the bearing capacity of 400KG is achieved.
Efficient Discrete Construction: An Experimental Design-to-Fabrication Workflow with Automatic UAV Integration
AI FABZhe Guo*, Zixun Huang*, Xuhui Lin*, Kai Xiao*, Sijie Gao*, Ziyue Hu, Xiaoliang Ying, Yitian Lu, You Lyu, Qixin Li, Lihua Zhang, Likai Wei, Hongxin Wang, Zihao Zhang, supervised by Prof. Xiang Wang*
[video] [poster] drawing credit: zczlxl3@ucl.ac.uk
❗We integrated drone tracking (via MoCap) and controlling (via ROS) into Grasshopper3D; Achieved discrete stacking with utilizing a UAV gripping system crafted from Raspberry Pi, PX4 and 3D printed hardware, etc.
📃A Series: Towards Mass Customization Techniques in Casting Mold Production Using 6-Axis Robotic Arms
AI FABZixun Huang*, Kunshen Huang*, Zee Liang*, Sihan Wang, Weishun Xu, Prof. Raspall Felix
[IASS 2019] [SiGraDi 2020] [poster] [demo installation] [video3] This is a
continuing series of researches on efficient mass customization. * denotes equal contribution. To check other
students' work co-mentored by me and Zee Leong.
❗We developed a rapid 3D clay printing system and hot-wire cuttingsystem using high-torque stepper motors, Arduino, KUKA Robots, etc. 🎉I also enabled non-planar robotic printing on quadric surfaces!!
Advancing Autonomous Resin Printing in the Air: Shaping with Gravity
AI FABZixun Huang, supervised by Dr. Dan Luo
[video] for real-world data exploration.
We achieved 3D printing in the air with the resin solidifying and being molded by gravity drop; parameterized the robotic printing behavior with extrusion speed, motion speed and dwell time, etc. Then we evaluated the capacities of imitation learning and offline reinforcement learning on training an autonomy for the resin shaping with gravity.
Human-AI Interaction
+ MORE
📃Discovering the City Through an Urban Researcher's Lens: See What They See
AI FABZixun Huang, Prof. Hao Zheng, Hang Gao
[demo video] [poster] in submission to Ubicomp 2024
We are releasing "Urban Lens"!!🎉 This work provides the public a novel way to aware our urban environment in a professional but visible way. We invited over 100 urban designers to collaborate on metrics evaluating!!
📃Can Machine Learning Uncover Insights into Vehicle Travel Demand from Our Built Environment?
AI FABZixun Huang, supervised by Prof. Hao Zheng
[manuscript] [poster] in submission to Cities
We demonstrated that predicting people's travel demand is achievable through the observation of Points of Interest (POI) spatial distribution. 🎉Surprisingly, we found that people's activities at various times of the day are influenced by urban design—an effect that extends across different cities.
Built Environment
+ MORE
📃MARL: Multiscale Archetype Representation Learning for Urban Building Energy Modeling
AI FABXinwei Zhuang*, Zixun Huang*, Wentao Zeng, supervised by Prof. Luisa Caldas
[github] [ICCVW 2023] [bibtex] [poster]
🎉Our work was accepted at ICCV workshop 2023, CVAAD. We achieved over 100x less computation time on urban-scale building energy estimation with significantly more accurate results.
Building Your Dream Home: Just Like Crafting with LEGO
AI FABXiao Jin*, Zixun Huang*, Qianlong Zhao*, Hang Gao, Hanzhi Zhang, Qingyang Zong, supervised by Prof. Hao Zheng
[video1] [video2] [video3] executable file: [exe, apk]
We developed a modular building information management (BIM) system from 0 to 1; Enabled efficient and scalable structure customization with real-time 3D visualization; Achieved an immersive user experience built on Android using Unreal Engine and Blueprints.
Augmented Reality
+ MORE
MIT RealityHACK Winner: Crafting an Immersive XR Work Environment with Spatial Audio
AI FABZixun Huang*, Weiyu Feng*, Yuehui Du*, Yola Wu*, Qianchen Bao
[project page] [github]
🎉We won the MIT RealityHACK 2023, in the track of the Best Use of Spatial Audio.
Thanks to my teammates!!! Thanking Dolby.io and Snapdragon AR for their excellent SDKs.
📃Universal AR-Enhanced Interface for ROS: Enabling Multi-Type Robot Control
AI FABErin Fan*, Zixun Huang*, supervised by Dr. Allen Y. Yang
[project page] under construction
This is a minimal prototype for NASA Suits 2024. Welcome to our MDes Open Showcase, we will achieve real-time Mars Rover localization and test user experiences for the LMCC (Local Mission Control Console).
Project
<Waiting>Travel Demand & Built Environment:
AI FABML Engineering
Project
XR Working Environment Editor: MIT Reality HACK 2023 Winner
AI FABAR & Andriod Dev
Project
INSOME Cloud:
AI FABAndriod Dev
Project
Generative Adversarial City Sense Generation
AI FABWeb Dev
🎉Recognition
> Awards & Scholarships.
MIT Reality HACK 2023 Winner - Spatial Audio Track. Massachusetts Institute of Technology, MA, USA. 2023
MDes Distinguished Scholar Award. Jacobs Institute of Design Innovation, University of California, Berkeley, CA, USA. 2022
Excellent in SRTP (Student Research Training Project). School of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China. 2018
ZJU Merit-based Scholarship. School of Material Science and Engineering, Zhejiang University, Hangzhou, China. 2016
Provincial Third Division of the National High School Mathematics League. Zhejiang Province, China. 2015
> Invited Talks.
A010125: AI Architecture Before and After. Dept. of Architecture. Xi'an University of Architecture and Technology. Oct. 24th, 2023
Design@Large Panel: Landing a Research Position. Jacobs Institute of Design Innovation, University of California, Berkeley. Sep. 22nd, 2023
Architectural Robotics: From Design to Construction. College of Civil Engineering and Architecture, Zhejiang University. Nov. 20th, 2019
> Selected Press.
Shrine of Whatslove / Wutopia Lab. ArchDaily. Mar. 27th, 2019
China’s first all carbon-fiber structure designed architecture. Gooood. May. 13th, 2019
Discrete Elements Construction of Automatic UAV. SOHU. Aug. 16th, 2019
🙇Find Where am I
>> Contributed to Digital Twin Tracking Dataset (DTTD v1.0).
>> Co-first authored the DTTD v1.1 based on iPhone LiDAR camera and ARKit, in submission to WACV.
>> Achieved 44 scene annotation with over 13k frames using programmed Python & C++ toolkits.
ALGORITHM
>> Led the model design and training strategy of the coarse estimator for 6-Dof object tracking pipeline.
>> Achieved a coarse estimator with accuracy (ADDS 98.32%) on both YCBV & DTTD-iPhone Dataset using Transformer-based dual-encoder structure; conducted over 25 recorded ablation experiments.
>> Developed a building customization and management system from 0 to 1 built on Android.
>> Tools: Unreal Engine 4, Java Spring, GraphQL, Redis, Tencent Cloud, Nginx, Django 3 (Agile Prototyping), Android, MySQL
>> Led and prototyped a cost management and sustainability scoring system from 0 to 1, and generalized it into launched urban planning projects.
>> Tools: Java Spring, React.js, Redux, Redis, Tencent Cloud, Nginx, Django REST Framework (Agile Prototyping), MySQL
>> Conducted over 85-page policy study report on logistics network business based on 15-minute city accessibility, community commerce, and revitalization of aging neighborhoods.
Worked closely with Kaihang L. Weishun X. Zhiyuan X. Yunpeng W. et.al
>> Developed Python codes to collected and analyzed about 537k online data (POI, Weibo, Twitter, MaFengWo).
>> Supported the design team to discover data-driven evidences for old towns revitalization in 3 cities: Chengdu, Xiangshan, and Xianju; produced about 400k RMB in revenue for startup design studio.
Worked closely with Weishun X. et.al
>> Designed and fabricated the China's first all-carbon fiber pavilion: Jieyuan Shrine and a robotic weaving system using KUKA Robots and programmable 3D modeling.
>> Achieved the 4 meters high and 3.8 meters wide entire structure weaved with a continuous line of carbon-fiber. Density of the structure is controlled at 18KG per cubic meter and the bearing capacity of 400kg is achieved.
ALGORITHM
>> Planned and programmed the robotic weaving path and the simulation for 40% modules of the pavilion.
Worked closely with Tim L. Zee L. Yuhong H. Peiyi H.
>> Improved the success rate by 63.6% of the robotic hot-wire cutting pipeline by fixing the robotic path generation algorithm.
Worked closely with Hao M. Zee L. Peiyi H.
COMPSCI 294-196 GenAI & LLM (Ongoing)
COMPSCI 285 Reinforcement Learning(Ongoing)
COMPSCI 282 Design Deep Neural Net
COMPSCI 280 Computer Vision
COMPSCI 294-137 Immersive Computing (Audit)
COMPSCI 294-026 Computational Photography
>> Co-first authored an VQAE-based method for residential buildings' latent embedding and clustering.
>> Reduced the computation time by 133.7 times for NYC's residential energy consumption estimation.
Worked closely with Phd. Xinwei Z.
>> First-authored an attention-based method for travel demand time-serials prediction from the perspective of built environment.
APPLICATION
>> Designed and implemented an urban sense mapping system based on adversarial generation algorithms from 0 to 1.
>> Shorten the average urban quantitative survey cost from 3 days to 3 seconds per person among over 100 designers.
>> Tools: Django, React.js, PyTorch, Tencent Cloud, Nginx, uWSGI, MySQL.
Selected Course: Application of Wireless Network, Image Analysis and Processing, Fundamentals of Computer Science, C Programming, Architectural Robotics, Computer Aided Design, Parametric Design, Architectural Physics, Architectural Design, Engineering Economy, Building Codes.
>> Architectural Robotics.
>> Computational Design &Robotic Fabrication.
PUBLICATION
>> Co-first authored: Robotic Fabrication of Sustainable Hybrid Formwork with Clay and Foam for Concrete Casting.
FIRST CONTRIBUTION
>> Designed the entire design-to-manufacturing workflow and led students to conduct it; wrote the first edition of the paper.
>> Engineered the entire 3D clay printing tool; achieved the SOTA efficiency for concrete formwork in material sustainable utilization, time &labor costs.
Worked closely with Weishun X.
Selected Course: Calculus I, Calculus II, Calculus III, Linear Algebra, Ordinary Differential Equations, Inorganic &Analytical Chemistry, University Physics A, Descriptive Geometry, Engineering Graphics.
