3D Interactive Scenes with Gemini 3 + Three.js

Why Gemini for 3D Development

Three.js has a notoriously steep learning curve. The API surface is vast—hundreds of classes for geometries, materials, lights, cameras, and post-processing effects. Even experienced developers spend hours consulting documentation.

Gemini 3 changes this equation dramatically. Its multimodal capabilities mean you can describe a scene in natural language, sketch it on a napkin, or show a reference image—and get working Three.js code in return. Gemini understands spatial relationships, lighting physics, and material properties at a level that produces surprisingly sophisticated scenes.

The workflow is especially powerful with React Three Fiber (@react-three/fiber), which wraps Three.js in React's declarative paradigm. Gemini generates clean, component-based 3D code that's easy to iterate on.

Setting Up Your First Scene

Let's build an interactive 3D scene from scratch using Gemini as your co-pilot.

npm install three @react-three/fiber @react-three/drei

import { Canvas } from '@react-three/fiber'
import { OrbitControls, Environment } from '@react-three/drei'

export function Scene() {
  return (
    <Canvas camera={{ position: [3, 3, 3], fov: 50 }}>
      <ambientLight intensity={0.5} />
      <directionalLight position={[5, 5, 5]} />
      <OrbitControls />
      <Environment preset="sunset" />
      {/* Your 3D objects go here */}
    </Canvas>
  )
}

Materials, Lighting, and Textures

Gemini excels at generating realistic material configurations. Instead of manually tweaking roughness, metalness, and normal maps, describe the look you want:

"Create a glass material with slight refraction and frosted edges" — Gemini outputs a MeshPhysicalMaterial with correct transmission, roughness, ior, and thickness parameters.

"Add dramatic studio lighting with a warm key light and cool fill" — You get a three-point lighting setup with properly configured DirectionalLight, SpotLight, and RectAreaLight components.

For textures, Gemini can generate procedural textures using shaders or suggest appropriate texture maps from common sources. The key is being specific about the visual effect you want rather than the technical implementation.

Adding Interactivity

Static scenes are impressive, but interactive ones are memorable. Here's how to add user interaction.

function InteractiveBox() {
  const [hovered, setHovered] = useState(false)
  return (
    <mesh
      onPointerOver={() => setHovered(true)}
      onPointerOut={() => setHovered(false)}
      scale={hovered ? 1.2 : 1}
    >
      <boxGeometry />
      <meshStandardMaterial
        color={hovered ? '#ff6b6b' : '#4ecdc4'}
      />
    </mesh>
  )
}

Performance Optimization