How to Build an Image Processor with React and Transformers.js

Introduction

In this tutorial, I'll show you how to build a high-performance image processing application that runs entirely in the browser. We'll use React for the UI, wasm-vips for image processing, and Web Workers for handling heavy computations without blocking the main thread.

The Technology Stack

• React: For building the user interface
• wasm-vips: A powerful image processing library compiled to WebAssembly
• Web Workers: For handling CPU-intensive tasks in background threads
• TypeScript: For type safety and better development experience

Implementation Details

1. Setting Up the Worker

First, let's set up our Web Worker for handling image processing tasks. We'll use wasm-vips inside the worker to avoid blocking the main thread:

// vips.worker.ts
import type Vips from "wasm-vips";
import { VipsTaskQueue } from "~/lib/VipsTaskQueue";

const FORMAT_CONFIGS = {
  jpg: {
    extension: "jpg",
    mimeType: "image/jpeg",
    getOptions: (options?: ConvertOptions) => ({
      Q: options?.quality,
      strip: options?.strip ?? true,
      optimize_coding: options?.quality !== undefined ? options?.optimizeCoding ?? true : false,
      optimize_scans: options?.quality !== undefined ? true : false,
      interlace: options?.interlace,
    }),
  },
  // ... other format configs
};

async function convertImage(
  arrayBuffer: ArrayBuffer,
  format: string,
  options?: ConvertOptions,
  id?: string
) {
  if (!vipsInstance?.Image) {
    throw new Error("Vips not initialized");
  }

  const taskId = id || Math.random().toString(36).slice(2);
  
  // Process image using vips
  const image = await vipsInstance.Image.newFromBuffer(arrayBuffer);
  const config = FORMAT_CONFIGS[format];
  const buffer = await image.writeToBuffer(
    \`.\${config.extension}\`,
    config.getOptions(options)
  );
  
  return { buffer, size: buffer.length };
}

2. Main Application Component

Here's how we implement the main application component that handles file uploads and processing:

// index.tsx
export default function ImageProcessor() {
  const [files, setFiles] = useState<ImageFile[]>([]);
  const [isConverting, setIsConverting] = useState(false);
  const { convert, error } = useVipsWorker();

  const handleFilesSelected = (newFiles: File[]) => {
    const newImageFiles = newFiles.map((file) => ({
      id: nanoid(),
      file,
      targetFormats: ["jpeg"],
      originalSize: file.size,
      quality: "high" as const,
      lossless: true,
    }));
    setFiles((prev) => [...prev, ...newImageFiles]);
  };

  const convertImages = async () => {
    if (files.length === 0) return;
    setIsConverting(true);

    try {
      const convertPromises = files.flatMap((file) =>
        file.targetFormats.map(async (format) => {
          const { buffer, size } = await convert({
            file: file.file,
            format,
            options: {
              quality: QUALITY_SETTINGS[file.quality!].quality,
              lossless: file.lossless,
              strip: true,
            },
            onProgress: (percent) => {
              // Update progress
            },
          });
          
          // Update file state with converted buffer
          setFiles((prev) =>
            prev.map((f) =>
              f.id === file.id
                ? {
                    ...f,
                    convertedBuffers: {
                      ...f.convertedBuffers,
                      [format]: buffer,
                    },
                  }
                : f
            )
          );
        })
      );

      await Promise.all(convertPromises);
    } finally {
      setIsConverting(false);
    }
  };
}

3. Custom Hook for Worker Management

We'll create a custom hook to manage the worker lifecycle and communication:

// useVipsWorker.ts
export function useVipsWorker() {
  const [error, setError] = useState<string | null>(null);
  const workerRef = useRef<Worker | null>(null);

  useEffect(() => {
    const worker = new Worker(new URL("./workers/vips.worker.ts", import.meta.url));
    workerRef.current = worker;

    worker.postMessage({ type: "init" });

    return () => worker.terminate();
  }, []);

  const convert = async ({ file, format, options, onProgress }) => {
    const arrayBuffer = await file.arrayBuffer();
    
    return new Promise((resolve, reject) => {
      const id = nanoid();
      
      workerRef.current?.addEventListener("message", function handler(e) {
        const { type, result, error, progress } = e.data;
        
        if (e.data.id !== id) return;
        
        switch (type) {
          case "progress":
            onProgress?.(progress);
            break;
          case "complete":
            resolve(result);
            workerRef.current?.removeEventListener("message", handler);
            break;
          case "error":
            reject(error);
            workerRef.current?.removeEventListener("message", handler);
            break;
        }
      });

      workerRef.current?.postMessage(
        {
          type: "convert",
          payload: { arrayBuffer, format, options },
          id,
        },
        [arrayBuffer]
      );
    });
  };

  return { convert, error };
}

Performance Optimization

The key to achieving good performance is:

1. Web Workers: All heavy image processing is done in a separate thread
2. Task Queue: We implement a task queue in the worker to prevent memory issues
3. WebAssembly: Using wasm-vips gives us near-native performance
4. Transferable Objects: We use transferable ArrayBuffers to avoid copying large amounts of data

Error Handling and Progress Tracking

We implement comprehensive error handling and progress tracking:

// Error handling in worker
try {
  const result = await taskQueue.enqueue(task);
  self.postMessage({ type: "complete", result, id: taskId });
} catch (error) {
  self.postMessage({
    type: "error",
    error: error instanceof Error ? error.message : String(error),
    id: taskId,
  });
}

// Progress tracking
image.onProgress = (progress) => {
  self.postMessage({ type: "progress", progress, id: taskId });
};

Conclusion

This implementation shows how to build a modern, high-performance image processing application that runs entirely in the browser. By leveraging WebAssembly through wasm-vips and using Web Workers for heavy computations, we can process images quickly without blocking the main thread.

The complete source code is available at Kitt Tools, where you can also try out the live demo.

References

• React Documentation
• wasm-vips GitHub Repository
• Web Workers API
• WebAssembly