Pan-tilt-zoom (PTZ) cameras, which dynamically adjust their field of view (FOV), are pervasive in large-scale scenes, such as train stations, squares, and airports. In real scenarios, PTZ cameras are required to quickly make decisions informed about where to direct its focus through contextual cues from the surrounding environment. To achieve this goal, some researches project camera videos into three-dimensional (3D) models or panoramas and allow operators to perceive spatial relationships. However, these works face several challenges in terms of real-time processing, localization accuracy, and realistic reference. To address this problem, we propose a visual expansion and real-time calibration for PTZ cameras assisted by panoramic models. We attempt to meet the demand for real-time processing with a motion estimation model for a PTZ camera, to improve calibration performance of PTZ images with only two feature point pairs, and to provide a realistic environmental context through a panoramic model. We verify our methods on both public and our self-built test scene. It can be seen from the experimental results that our method can exhibit impressive accuracy and efficiency.

In this paper, we present an Augmented Reality (AR)-based application combined with a robotic system for percutaneous renal puncture navigation interaction and demonstrate its technical feasibility. Our system provides an intuitive interaction scheme between the surgeon and the robot without the need for traditional external input devices, and applies an image-target-based 3D registration scheme to transform the coordinate system between Hololens2 and the robot without using additional tracking devices. Users can visualise the abdominal puncture phantom and obtain 3D depth information of the lesion site by wearing Hololens2 and control the robot directly using buttons or gestures. To investigate the accuracy and feasibility of the proposed interaction scheme, six subjects were recruited to complete 3D registration alignment accuracy experiments, and puncture positioning accuracy experiments using ultrasound unaided navigation, AR unaided navigation and AR robotic navigation. The results showed that the average alignment error of 3D registration was 3.61±1.05 mm. The average positioning errors of ultrasound freehand navigation, AR freehand navigation and AR robotic navigation were 7.67±2.00 mm, 6.13±1.07 mm and 5.52±0.37 mm, respectively; the average puncture times were 34.86±1.67 s, 22.40±2.07 s, and 29.41±1.37 s.

Stylized animation is often adored for its innovative and daring visual creativity. Due to the strong visual impact and color contrast inherent in woodcut style design, it has been applied in animation and comics. However, traditional woodcuts, hand-drawn, and previous computer-aided methods have yet to address the issues of dwindling design inspiration, lengthy production times, and complex adjustment procedures. To tackle these challenges, we propose a novel network framework, the Woodcut-style Design Assistant Network (WDANet). Notably, our research is the first to utilize diffusion models to streamline the woodcut-style design process. We curate the Woodcut-62 dataset, which features works from 62 renowned historical artists, to train WDANet in absorbing and learning the aesthetic nuances of woodcut prints, offering users a wealth of design references. Our WDANet integrates text and woodcut-style image features based on a denoising network. WDANet allows users to input or slightly modify a text description to quickly generate accurate, high-quality woodcut-style designs, saving time and offering flexibility. As confirmed by user studies, quantitative and qualitative analyses show that WDANet outperforms the current state-of-the-art in generating woodcut-style images and proves its value as a design aid.

As virtual reality display technologies advance, resolutions and refresh rates continue to approach human perceptual limits, presenting a challenge for real-time rendering algorithms. Neural super-resolution is promising in reducing the computation cost and boosting the visual experience by scaling up low-resolution renderings. However, the added workload of running neural networks cannot be neglected. In this paper, we try to alleviate the burden by exploiting the foveated nature of the human visual system, where acuity decreases rapidly from the focal point to the periphery. With the help of dynamic and geometric information (i.e.,pixel-wise motion vectors, depth, and camera transformation) available inherently in the real-time rendering content, we propose a neural accumulator to effectively aggregate the amortizedly rendered low-resolution visual information from frame to frame recurrently. By leveraging a partition-assemble scheme, we use a neural super-resolution module to upsample the low-resolution image tiles to different qualities according to their perceptual importance and reconstruct the final output heterogeneously. Perceptually high-fidelity foveated high-resolution frames are generated in real-time, surpassing the quality of other foveated super-resolution methods.

：Explore the digital rendering mode with texturing brush and ink effect, analyze and simulate the brush and ink characteristics of traditional Chinese ink painting, especially the art of landscape painting, and try to integrate the texture of wrinkle method synthesis map and modeling technology based on particle deposition and stacking (overlapping) to render, thus realizing the computer simulation of small freehand brushwork and large freehand brushwork (splash ink) ink effects in traditional landscape painting. This research inherits and develops the aesthetic theory and thought of traditional ink painting art, and makes some beneficial exploration in the field of digital freehand ink painting, which has positive reference value and promotion significance for the development of Chinese ink painting.

Most existing makeup transfer techniques focus on light makeup styles, which limits the task of makeup transfer to color manipulation issues such as eye shadow and lip gloss. However, the makeup in real life is diverse and personalized, not only the most basic foundation, eye makeup, but also the painted patterns on the face, jewelry decoration and other personalized makeup. Inspired by the painting steps of drawing the outline first and then coloring, we propose a makeup transfer network for personalized makeup, which realizes face makeup transfer by learning outline correspondence. Specifically, we propose the outline feature extraction module and outline loss that can promote outline correspondence. Our network can not only transfer daily light makeup, but also complete transfer for complex facial painting patterns. Experiments show that our method can obtain visually more accurate makeup transfer results. Quantitative and qualitative experimental results show that the method proposed in this paper achieves superior results in extreme makeup transfer compared to the state-of-the-art methods.

Redirected walking (RDW) provides an immersive user experience in virtual reality applications. In RDW, the size of the physical play area is limited, which makes it challenging to design the virtual path in a larger virtual space. Mainstream RDW approaches rigidly manipulate gains to guide the user to follow predetermined rules. However, these methods may cause simulator sickness, boundary collision, and reset. Static mapping approaches warp the virtual path through expensive vertex replacement in the stage of model pre-processing. They are restricted to narrow spaces with non-looping pathways, partition walls, and planar surfaces. These methods fail to provide a smooth walking experience for large-scale open scenes. To tackle these problems, we propose a novel approach that dynamically redirects the user to walk in a non-linear virtual space. More specifically, we propose a Bezier-curve-based mapping algorithm to warp the virtual space dynamically and apply multiperspective fusion for visualization augmentation. We conduct comparable experiments to show its superiority over state-of-the-art large-scale redirected walking approaches on our self-collected photogrammetry dataset.

In order to be successfully executed, collaborative tasks performed by two agents often require a cooperative strategy to be learned. In this work, we propose a constraint-based multi-agent reinforcement learning approach called Constrained Multi-agent Soft Actor Critic (C-MSAC) to train control policies for simulated agents performing collaborative multi-phase tasks. Given a task with n phases, the first n-1 phases are treated as constraints for the final task phase objective, which is addressed with a centralized training and decentralized execution approach. We highlight our framework on a tray balancing task including two phases: tray lifting and cooperative tray control for target following. We evaluate our proposed approach and compare it against its unconstrained variant (MSAC). The performed comparisons show that C-MSAC leads to higher success rates, more robust control policies, and better generalization performance.

The particular characteristics of deaf-mutes make them more likely to have mental health problems. Due to their particular way of communication, it is more difficult for them to deal with mental health problems than ordinary people. Nowadays, those psychologists who are good at sign language are in short supply, and remote assistance cannot achieve satisfactory results. Therefore, a library of virtual emotional gestures based on electroencephalogram(EEG) was established and a prototype system for mental health examination of deaf-mutes was proposed, which help deaf-mutes identify their psychological problems in time and assist medical staff to examine the psychological problems encountered by deaf-mutes. In addition, the virtual library of emotional gestures is established with the assistance of the chief physician from a 3A hospital in Henan Province. More importantly, the later experiments demonstrate the applicability of this virtual system.

Virtual Reality (VR) and Augmented Reality (AR) applications are becoming increasingly prevalent. However, constructing realistic 3D hands, especially when two hands are interacting, from a single RGB image remains a major challenge due to severe mutual occlusion and the enormous diversity of hand poses. In this paper, we propose a Disturbing Graph Contrastive Learning strategy for two-hand 3D reconstruction. This involves a graph disturbance network designed to generate graph feature pairs to enhance the consistency of the two-hand pose features. A contrastive learning module leverages high-quality generative features for a strong feature expression. We further propose a similarity distinguish method to divide positive and negative features for accelerating the model convergence. Additionally, a multi-term loss is designed to balance the relation among the hand pose, the visual scale and the viewpoint position. Our model has achieved State-of-the-Art results in the InterHand2.6M benchmark. Ablation studies show the model’s great ability to correct unreasonable hand movements. In subjective assessments, our Graph Disturbance Learning method significantly improves the construction of realistic 3D hands, especially when two hands are interacting.

Desiccation cracking of soil-like materials is a common phenomenon in natural dry environment, however, it remains a challenge to model and simulate complicated multi-physical processes inside the porous structure. With the goal of tracking such physical evolution accurately, we propose an MPM based method to simulate volumetric shrinkage and crack during moisture diffusion. At the physical level, we introduce Richards equations to evolve the dynamic moisture field to model evaporation and diffusion in unsaturated soils, with which a elastoplastic model is established to simulate strength changes and volumetric shrinkage via a novel saturation-based hardening strategy during plastic treatment. At the algorithmic level, we develop an MPM-fashion numerical solver for the proposed physical model and achieve stable yet efficient simulation towards delicate deformation and fracture. At the geometric level, we propose a correlating stretching criteria and a saturation-aware extrapolation scheme to extend existing surface reconstruction for MPM, producing visual compelling soil appearance. Finally, we manifest realistic simulation results based on the proposed method with several challenging scenarios, which demonstrates usability and efficiency of our method.

A great number of virtual museums exhibit archaeological artefacts in an interactive form using 3D digital models to disseminate them to as many users as possible. Currently, 3D models obtained as a result of scanning museum objects have gained a great popularity. This technique allows to faithfully reproduce objects and to expose artefacts too delicate and precious to be presented in the real world. The aim of this paper is to create a virtual museum in the VR world using 3D models obtained by 3D scanning with structured light.  The advantages of optimising mesh models for their use in virtual exhibitions are discussed. Unity and Unreal Engine, were used to create virtual museums in the VR world. Two twin test applications were prepared. This allowed to compare the performance of the developed applications (processor, graphics card and RAM load). A survey was also conducted among the users of the implemented virtual museum application. For immersion in the world of VR, a low-cost solution was used, involving the use of cheap VR frames, a smartphone and a computer. The frames together with the smartphone were used to display the image, while the entire rendering process was performed on the computer.

In this paper, we propose an integrated model for simulating the interaction between crowds and fluid particles. Our focus is on simulating evacuation motion for crowds in the face of sudden floods. Our model treats both the crowd and the water as fluid particles, which allows us to incorporate various forces such as pressure, shear, buoyancy, and active forces to drive the agents. Additionally, we have designed a minimum rotational path-planning algorithm for agents to search for safe destinations during evacuations. To develop practical crowd evacuation strategies, we observed and studied survival techniques from whirlpools and sudden changes in water levels during floods. Our simulated evacuation results provide plausible strategies for crowds to survive dangerous floods.

This paper presents a novel real-time facial feature extraction algorithm, producing a small feature set, suitable for implementing emotion recognition with online game and metaverse avatars. The algorithm aims to reduce data transmission and storage requirements, hurdles in the adoption of emotion recognition in these mediums. The early results presented show a facial emotion recognition accuracy of up to 92% on one benchmark dataset, with an overall accuracy of 77.2% across a wide range of datasets, demonstrating the early promise of the research.