Technical Explanation and System Features
技術解說與系統特色
將傳統的操偶手勢進行數位化
———————————————————————
該手勢設計是與布袋戲師傅共同合作開發的，涵蓋了傳統布袋戲中的五大角色類型：生、旦、淨、丑、雜。並基於傳統的經典手勢，開發了一個手勢識別系統，該系統包含10個主要手勢，每個手勢都有對應的視覺效果、物體互動和場景設計。
Digitizing traditional puppeteering gestures.
———————————————————————
The gesture design was developed in collaboration with master puppeteers and covers the five major character types in traditional Taiwanese puppetry: Sheng, Dan, Jing, Chou, and Za. Based on classic traditional gestures, a gesture recognition system was created, incorporating 10 primary gestures. Each gesture is associated with specific visual effects, object interactions, and scene designs.
數位化的操偶更加容易學習並增強遊玩的效果
———————————————————————
生的甩袖動作通過線條效果進行強化，以模擬微風的視覺效果。旦的行走動作加入了花朵生長和飄落的效果，使角色顯得更加優雅。淨的轉身動作融合了動態披風的動作和整體木偶的旋轉，這是一個相對複雜的動作，但在VR中能輕鬆觸發。丑的逃跑動作，在傳統布袋戲中只能象徵性地表現出來，但在我們的系統中實際上可以實現，允許角色從使用者的掌控中逃脫。雜的後空翻動作，這是現實中最具挑戰性的動作之一，需要大量的練習才能掌握，但在VR中變得幾乎不用學習就可以掌握，並且通過視覺效果和互動使整個過程更加有趣。
The digitizing puppeteering is easier to learn and enhances the overall gameplay experience.
———————————————————————
The "Sheng" character's sleeve flick is enhanced with line effects to simulate the visual effect of a gentle breeze. The "Dan" character's walking motion is accompanied by the growth and falling of flowers, adding an elegant touch to the character. The "Jing" character's turning motion combines the dynamic movement of a flowing cape with the full rotation of the puppet—a relatively complex action that can be easily triggered in VR. The "Chou" character's escape motion, traditionally only symbolically represented in Taiwanese puppetry, is fully realized in our system, allowing the character to actually escape from the user's control. The "Za" character's backflip, one of the most challenging moves in real life that requires extensive practice to master, becomes almost effortless in VR, with added visual effects and interactions making the entire process more engaging and fun.
透過IOT實現不同的操偶串聯
———————————————————————
我們利用大多數人都擁有的智慧型手機作為互動媒介，整合到IoT平台，實現跨裝置互動體驗，提出了一套基於CAVE系統的沈浸式多人操偶系統，進一步探究以空間沈浸式顯示器為主的虛擬實境對於多人操偶體驗的影響。
Implementing different puppeteering through IoT integration.
———————————————————————
We utilize the widely available smartphones as interactive mediums, integrating them into an IoT platform to enable cross-device interaction. We have developed an immersive multi-user puppet manipulation system based on the CAVE system, further exploring the impact of spatial immersive displays in virtual reality on the multi-user puppet manipulation experience.
透過手機操偶實現多人沉浸式互動
———————————————————————
本研究使用遊戲引擎Unity 進行開發，利用智慧型手機內建的加速度計與陀螺儀之數值，操控虛擬戲偶的方向（Orientation）與動畫。在方向的控制上，將手機陀螺儀三個軸向的旋轉角度，直接套用在虛擬戲偶的三個旋轉軸向上。而在動畫控制上，我們整合了陀螺儀與加速度計的數值，將實際操偶的手勢進行簡化，設計了一系列基於手機的手勢偵測。當使用者以特定的方式旋轉手機、揮舞手機、點擊手機，其手勢對應的動畫就會觸發，例如左右搖擺手機會觸發逃跑的動畫，虛擬戲偶會向前跑；兩根手指點擊螢幕則會觸發漫步的動畫。
Achieving multi-user immersive interaction through smartphone-controlled puppet manipulation.
———————————————————————
This study was developed using the Unity game engine, leveraging the built-in accelerometer and gyroscope of smartphones to control the orientation and animations of virtual puppets. For orientation control, the rotation angles of the smartphone's gyroscope across three axes are directly applied to the three rotational axes of the virtual puppet. In terms of animation control, we integrated the data from the gyroscope and accelerometer to simplify the actual puppet manipulation gestures, designing a series of smartphone-based gesture detections. When users rotate, wave, or tap their smartphones in specific ways, corresponding animations are triggered. For example, shaking the phone from side to side triggers the escape animation, causing the virtual puppet to run forward, while a two-finger tap on the screen triggers a walking animation.