init

hy3dshape/tools/README.md

@@ -0,0 +1,94 @@
+# Data Processing
+
+This is the data processing pipeline for 3D shape and texture generation.
+
+**Notes**:
+1. This implementation is a simplified version of our industrial pipeline.
+2. The rendering script is based on [TRELLIS](https://github.com/microsoft/TRELLIS/blob/main/dataset_toolkits/blender_script/render.py).
+
+## Rendering
+
+### Motivation
+The rendering script `render/render.py` serves three main purposes:
+1. Converting complex 3D formats to PLY files using Blender for further processing.
+2. Rendering condition images for DiT training.
+3. Rendering orthogonal images, PBR materials, and conditional signals (world-space normals and positions) for texture generation.
+
+### Requirements
+The rendering scripts are executed with Blender 4.1. You need to install `opencv`, `OpenEXR`, and `Imath` using Blender's Python. Here is an example for a Macbook:
+```bash
+/Applications/Blender.app/Contents/Resources/4.1/python/bin/python3.11 -m pip install OpenEXR Imath opencv-python
+```
+
+### Execution
+The first two purposes can be executed with a single command:
+```bash
+$BLENDER_PATH -b -P render/render.py -- \
+    --object ${INPUT_FILE} --geo_mode --resolution 512 \
+    --output_folder $OUTPUT_FOLDER
+```
+For the third purpose, simply remove the `--geo_mode` flag.
+
+## Watertight Mesh Processing and Sampling
+
+### Motivation
+To learn an SDF representation for 3DShape2VecSets, we require a watertight input mesh. This pipeline processes raw triangle meshes to generate three essential data types:
+1. **Surface samples** - Input points for the encoder.
+2. **Volume samples** - Query points for SDF evaluation in the decoder.
+3. **Volume SDFs** - Ground-truth signed distance values for VAE training.
+
+### Execution
+Process a triangle mesh (OBJ/OFF format) to generate:
+1. Watertight mesh (`${OUTPUT_NAME}_watertight.obj`).
+2. Surface point samples (`${OUTPUT_NAME}_surface.npz`).
+3. Volume samples with SDFs (`${OUTPUT_NAME}_sdf.npz`).
+
+**Command:**
+```bash
+python3 watertight/watertight_and_sample.py \
+    --input_obj ${INPUT_MESH} \
+    --output_prefix ${OUTPUT_NAME}
+```
+
+### Output Data Format
+
+#### 1. Surface Samples (`${OUTPUT_NAME}_surface.npz`)
+Contains two point cloud arrays in numpy NPZ format:
+
+| Key             | Shape    | Format   | Description                     |
+|-----------------|----------|----------|---------------------------------|
+| `random_surface` | `(N, 6)` | `float16`| Uniform point samples on surface |
+| `sharp_surface`  | `(M, 6)` | `float16`| Samples near sharp mesh edges   |
+
+#### 2. Volume SDF Samples (`${OUTPUT_NAME}_sdf.npz`)
+Contains three sample types stored as array pairs. For each type `${type}`:
+
+| Sample Type     | Points Array         | SDF Labels Array     | Shape    | Format   | Description             |
+|-----------------|----------------------|----------------------|----------|----------|-------------------------|
+| `vol`          | `vol_points`        | `vol_label`         | `(P, 3)/(P,)` | `float16`| Random spatial samples |
+| `random_near`   | `random_near_points` | `random_near_label`  | `(Q, 3)/(Q,)` | `float16`| Samples near surface   |
+| `sharp_near`    | `sharp_near_points`  | `sharp_near_label`   | `(R, 3)/(R,)` | `float16`| Samples near sharp edges |
+
+**Data Specifications**:
+- All point coordinates (`*_points` arrays) contain 3D positions stored as `float16` values.
+- All SDF values (`*_label` arrays) are `float16` scalars representing:
+  - **Positive values**: Outside the surface.
+  - **Negative values**: Inside the surface.
+  - **Zero values**: On the surface.
+- Array dimensions:
+  - `N`, `M`, `P`, `Q`, `R` represent sample counts (vary per shape).
+  - `3` indicates XYZ coordinates.
+  - `6` indicates XYZ/Normal coordinates.
+- All arrays are stored uncompressed in numpy's NPZ format.
+
+## Overall Script
+Modify the first four variables in `pipeline.sh`:
+1. **INPUT_FILE** The path to each 3D data file.
+2. **OUTPUT_FOLDER** The overall path for the output dataset.
+3. **NAME** The naming for the output path of each data.
+4. **BLENDER_PATH** The executable path for Blender.
+
+Then run the following script:
+```bash
+bash pipeline.sh
+```