Skip to content

api-media

Vue d'ensemble

Le service api-media gère le stockage, l'upload et la gestion des médias de la plateforme Roomee. Il utilise Google Cloud Storage (GCP) pour le stockage des fichiers et offre des fonctionnalités avancées comme la génération de thumbnails et la conversion de formats.

CaractéristiqueDétail
Port3003
StackJavaScript (Legacy), Express.js, GCP Storage
Base de donnéesMongoDB (minimal - legacy notifications schema)
ArchitectureLegacy (JS + GCP SDK)
Complexité⭐⭐⭐

Responsabilités

mermaid
graph TB
    MEDIA[api-media]

    subgraph "Fonctionnalités"
        UPLOAD[Upload Fichiers]
        PRESIGNED[Presigned URLs]
        THUMBNAIL[Génération Thumbnails]
        CONVERSION[Conversion Formats]
        QUOTA[Gestion Quotas]
        EDITOR[Editor.js Integration]
    end

    MEDIA --> UPLOAD
    MEDIA --> PRESIGNED
    MEDIA --> THUMBNAIL
    MEDIA --> CONVERSION
    MEDIA --> QUOTA
    MEDIA --> EDITOR

    subgraph "Stockage"
        GCP[Google Cloud Storage]
        BUCKETS[Buckets par Écosystème]
    end

    UPLOAD --> GCP
    PRESIGNED --> GCP
    THUMBNAIL --> GCP
    CONVERSION --> BUCKETS

Structure du Projet

api-media/
├── app.js
├── loader.js
├── controllers/
│   ├── media.controller.js         # Upload et gestion médias
│   ├── storage.controller.js       # Gestion quotas
│   └── editor.controller.js        # Editor.js integration
├── routes/
│   ├── media.routes.js
│   ├── storage.routes.js
│   └── editor.routes.js
├── services/
│   ├── gcp.service.js              # Google Cloud Storage
│   ├── thumbnail.service.js        # Sharp pour thumbnails
│   ├── conversion.service.js       # Conversion de formats
│   └── quota.service.js            # Gestion des quotas
├── middlewares/
│   ├── auth.middleware.js
│   ├── upload.middleware.js        # Multer configuration
│   └── quota.middleware.js         # Vérification quota
├── utils/
│   ├── ApiError.js
│   ├── catchAsync.js
│   └── fileHelper.js               # Helpers fichiers
├── amqp/
│   └── storageConsumer.js          # Consumer storage.created
├── config/
│   └── config.js
└── __tests/

Architecture d'Upload

Upload Direct vers GCP (Presigned URLs)

mermaid
sequenceDiagram
    participant Client
    participant API as api-media
    participant GCP as Google Cloud Storage

    Client->>API: POST /media/generate-presigned-url
    API->>GCP: Generate signed URL (15min TTL)
    GCP-->>API: Signed URL
    API-->>Client: { url, metadata }

    Client->>GCP: PUT (direct upload)
    GCP-->>Client: Upload success

    Client->>API: POST /media/upload-complete
    API->>GCP: Get file metadata
    API->>API: Generate thumbnail
    API->>GCP: Upload thumbnail
    API-->>Client: { fileUrl, thumbnailUrl }

Avantages :

  • ✅ Pas de transit par le serveur (économie bande passante)
  • ✅ Upload direct client → GCP (meilleure performance)
  • ✅ Scalabilité maximale
  • ✅ Réduction de la charge serveur

Routes API

📤 Upload Médias (/media)

POST /media/generate-presigned-url

Générer une URL signée pour upload direct vers GCP.

Headers :

Authorization: Bearer eyJhbGciOiJI...

Request :

json
{
  "filename": "photo-hotel.jpg",
  "contentType": "image/jpeg",
  "size": 2048576,
  "ecosystemId": "65f1111111111111111111111",
  "folder": "photos"
}

Response :

json
{
  "uploadUrl": "https://storage.googleapis.com/roomee-storage/...",
  "fileUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/photo-hotel.jpg",
  "expiresAt": "2025-01-21T10:15:00.000Z",
  "metadata": {
    "filename": "photo-hotel.jpg",
    "contentType": "image/jpeg",
    "size": 2048576,
    "uploadId": "upload-abc123"
  }
}

Logique :

javascript
const { Storage } = require('@google-cloud/storage')
const storage = new Storage({
  projectId: process.env.GCP_PROJECT_ID,
  keyFilename: process.env.GCP_KEY_FILE
})

const bucket = storage.bucket(process.env.GCP_BUCKET_NAME)
const filePath = `ecosystems/${ecosystemId}/${folder}/${filename}`

const [url] = await bucket.file(filePath).getSignedUrl({
  version: 'v4',
  action: 'write',
  expires: Date.now() + 15 * 60 * 1000, // 15 minutes
  contentType: contentType
})

return { uploadUrl: url, fileUrl: `https://.../${filePath}` }

POST /media/upload-complete

Confirmer la fin d'upload et générer le thumbnail.

Request :

json
{
  "uploadId": "upload-abc123",
  "fileUrl": "https://storage.googleapis.com/roomee-storage/...",
  "generateThumbnail": true
}

Response :

json
{
  "fileUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/photo-hotel.jpg",
  "thumbnailUrl": "https://storage.googleapis.com/roomee-storage/ecosystems/65f.../photos/thumbnails/photo-hotel-thumb.jpg",
  "metadata": {
    "size": 2048576,
    "width": 1920,
    "height": 1080,
    "format": "jpeg"
  }
}

Logique :

  1. Vérifier que le fichier existe sur GCP
  2. Récupérer les métadonnées du fichier
  3. Si image, générer le thumbnail (Sharp)
  4. Uploader le thumbnail vers GCP
  5. Mettre à jour le quota utilisé

POST /media/upload/

Upload direct via le serveur (fallback).

Headers :

Authorization: Bearer eyJhbGciOiJI...
Content-Type: multipart/form-data

FormData :

  • file : Fichier à uploader
  • ecosystemId : ID de l'écosystème
  • folder : Dossier destination (optionnel)

Response :

json
{
  "fileUrl": "https://storage.googleapis.com/...",
  "thumbnailUrl": "https://storage.googleapis.com/...",
  "metadata": {
    "filename": "photo-hotel.jpg",
    "size": 2048576,
    "contentType": "image/jpeg"
  }
}

Limites :

  • Taille max : 50 MB
  • Formats supportés : Images (JPEG, PNG, WebP, GIF), Vidéos (MP4, WebM), Documents (PDF)

POST /media/upload-chunk

Upload par chunks (pour mobile et gros fichiers).

Request :

json
{
  "chunkIndex": 0,
  "totalChunks": 5,
  "uploadId": "upload-abc123",
  "chunk": "<base64_data>"
}

Response :

json
{
  "uploadId": "upload-abc123",
  "chunkIndex": 0,
  "received": true,
  "progress": 20
}

Logique :

  1. Stocker le chunk temporairement
  2. Quand tous les chunks sont reçus, reconstituer le fichier
  3. Uploader vers GCP
  4. Nettoyer les chunks temporaires

DELETE /media/delete/:fileUrl

Supprimer un fichier de GCP.

Response :

json
{
  "message": "File deleted successfully",
  "fileUrl": "https://storage.googleapis.com/..."
}

GET /media/download/:ecosystemId/:filename

Télécharger un fichier (avec URL signée temporaire).

Response :

json
{
  "downloadUrl": "https://storage.googleapis.com/...",
  "expiresAt": "2025-01-21T11:00:00.000Z"
}

🖼️ Thumbnails (/media/thumbnail)

POST /media/thumbnail/generate

Générer un thumbnail pour une image.

Request :

json
{
  "fileUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
  "width": 300,
  "height": 300,
  "quality": 80
}

Response :

json
{
  "thumbnailUrl": "https://storage.googleapis.com/roomee-storage/thumbnails/photo-thumb.jpg",
  "width": 300,
  "height": 300,
  "size": 45678
}

Logique (Sharp) :

javascript
const sharp = require('sharp')

// Télécharger l'image depuis GCP
const imageBuffer = await downloadFromGCP(fileUrl)

// Générer le thumbnail
const thumbnailBuffer = await sharp(imageBuffer)
  .resize(width, height, {
    fit: 'cover',
    position: 'center'
  })
  .jpeg({ quality })
  .toBuffer()

// Uploader le thumbnail vers GCP
const thumbnailUrl = await uploadToGCP(thumbnailBuffer, thumbnailPath)

return { thumbnailUrl }

POST /media/thumbnail/batch

Générer des thumbnails pour plusieurs images.


🔄 Conversion de Formats (/media/convert)

POST /media/convert/to-webp

Convertir une image en WebP (format optimisé).

Request :

json
{
  "fileUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
  "quality": 85
}

Response :

json
{
  "originalUrl": "https://storage.googleapis.com/roomee-storage/photo.jpg",
  "convertedUrl": "https://storage.googleapis.com/roomee-storage/photo.webp",
  "originalSize": 2048576,
  "convertedSize": 512000,
  "savings": "75%"
}

POST /media/convert/to-jpeg

Convertir une image en JPEG.


📦 Gestion des Quotas (/storage)

POST /storage/create

Créer un quota de stockage pour un écosystème (appelé par api-hotel via AMQP).

Request :

json
{
  "ecosystemId": "65f1111111111111111111111",
  "maxSize": 20000,
  "unit": "MO"
}

Response :

json
{
  "ecosystemId": "65f1111111111111111111111",
  "maxSize": 20000,
  "unit": "MO",
  "used": 0,
  "available": 20000,
  "percentage": 0
}

GET /storage/get/:ecosystemId

Récupérer le quota d'un écosystème.

Response :

json
{
  "ecosystemId": "65f1111111111111111111111",
  "maxSize": 20000,
  "used": 5432,
  "available": 14568,
  "percentage": 27.16,
  "files": [
    {
      "path": "ecosystems/65f.../photos/photo1.jpg",
      "size": 2048,
      "createdAt": "2025-01-20T10:00:00.000Z"
    }
  ]
}

PATCH /storage/update/:ecosystemId

Mettre à jour le quota (augmentation/diminution).


✏️ Editor.js Integration (/editor)

POST /editor/upload-by-url

Uploader une image via URL (pour Editor.js).

Request :

json
{
  "url": "https://example.com/image.jpg",
  "ecosystemId": "65f1111111111111111111111"
}

Response :

json
{
  "success": 1,
  "file": {
    "url": "https://storage.googleapis.com/roomee-storage/...",
    "width": 1920,
    "height": 1080
  }
}

POST /editor/upload-by-file

Uploader une image via fichier (drag & drop Editor.js).

FormData :

  • image : Fichier image
  • ecosystemId : ID écosystème

Response (Format Editor.js) :

json
{
  "success": 1,
  "file": {
    "url": "https://storage.googleapis.com/..."
  }
}

Services

GCP Service

javascript
// services/gcp.service.js
const { Storage } = require('@google-cloud/storage')

class GCPService {
  constructor() {
    this.storage = new Storage({
      projectId: process.env.GCP_PROJECT_ID,
      keyFilename: process.env.GCP_KEY_FILE
    })
    this.bucket = this.storage.bucket(process.env.GCP_BUCKET_NAME)
  }

  async uploadFile(filePath, fileBuffer, contentType) {
    const file = this.bucket.file(filePath)

    await file.save(fileBuffer, {
      contentType: contentType,
      metadata: {
        cacheControl: 'public, max-age=31536000'
      }
    })

    return file.publicUrl()
  }

  async deleteFile(filePath) {
    const file = this.bucket.file(filePath)
    await file.delete()
  }

  async getSignedUrl(filePath, action, expiresIn) {
    const file = this.bucket.file(filePath)

    const [url] = await file.getSignedUrl({
      version: 'v4',
      action: action, // 'read' or 'write'
      expires: Date.now() + expiresIn
    })

    return url
  }

  async getFileMetadata(filePath) {
    const file = this.bucket.file(filePath)
    const [metadata] = await file.getMetadata()

    return {
      size: metadata.size,
      contentType: metadata.contentType,
      createdAt: metadata.timeCreated,
      updatedAt: metadata.updated
    }
  }

  async listFiles(prefix) {
    const [files] = await this.bucket.getFiles({ prefix })

    return files.map(file => ({
      name: file.name,
      size: file.metadata.size,
      contentType: file.metadata.contentType,
      createdAt: file.metadata.timeCreated
    }))
  }
}

Thumbnail Service

javascript
// services/thumbnail.service.js
const sharp = require('sharp')

class ThumbnailService {
  async generateThumbnail(imageBuffer, options = {}) {
    const {
      width = 300,
      height = 300,
      quality = 80,
      format = 'jpeg'
    } = options

    const thumbnail = await sharp(imageBuffer)
      .resize(width, height, {
        fit: 'cover',
        position: 'center'
      })
      .toFormat(format, { quality })
      .toBuffer()

    return thumbnail
  }

  async generateMultipleSizes(imageBuffer, sizes) {
    const thumbnails = {}

    for (const [name, { width, height }] of Object.entries(sizes)) {
      thumbnails[name] = await this.generateThumbnail(imageBuffer, {
        width,
        height
      })
    }

    return thumbnails
  }

  async getImageMetadata(imageBuffer) {
    const metadata = await sharp(imageBuffer).metadata()

    return {
      width: metadata.width,
      height: metadata.height,
      format: metadata.format,
      size: metadata.size
    }
  }
}

// Tailles prédéfinies
const THUMBNAIL_SIZES = {
  small: { width: 150, height: 150 },
  medium: { width: 300, height: 300 },
  large: { width: 600, height: 600 }
}

Conversion Service

javascript
// services/conversion.service.js
const sharp = require('sharp')

class ConversionService {
  async convertToWebP(imageBuffer, quality = 85) {
    return await sharp(imageBuffer)
      .webp({ quality })
      .toBuffer()
  }

  async convertToJPEG(imageBuffer, quality = 90) {
    return await sharp(imageBuffer)
      .jpeg({ quality })
      .toBuffer()
  }

  async convertToPNG(imageBuffer) {
    return await sharp(imageBuffer)
      .png()
      .toBuffer()
  }

  async optimizeImage(imageBuffer, format) {
    const options = {
      jpeg: { quality: 85, progressive: true },
      png: { compressionLevel: 9 },
      webp: { quality: 85 }
    }

    return await sharp(imageBuffer)
      .toFormat(format, options[format])
      .toBuffer()
  }
}

Quota Service

javascript
// services/quota.service.js
class QuotaService {
  async getQuota(ecosystemId) {
    // Récupérer le quota depuis la base
    const quota = await prisma.storage.findUnique({
      where: { ecosystemId }
    })

    if (!quota) {
      throw new ApiError(404, 'Quota not found')
    }

    // Calculer l'utilisation réelle depuis GCP
    const files = await gcpService.listFiles(`ecosystems/${ecosystemId}`)
    const used = files.reduce((sum, file) => sum + file.size, 0)

    return {
      ecosystemId,
      maxSize: quota.maxSize,
      used: Math.round(used / (1024 * 1024)), // MB
      available: quota.maxSize - Math.round(used / (1024 * 1024)),
      percentage: (used / (quota.maxSize * 1024 * 1024)) * 100,
      files: files.length
    }
  }

  async checkQuota(ecosystemId, fileSize) {
    const quota = await this.getQuota(ecosystemId)
    const fileSizeMB = fileSize / (1024 * 1024)

    if (quota.available < fileSizeMB) {
      throw new ApiError(413, 'Storage quota exceeded')
    }

    return true
  }

  async updateUsage(ecosystemId, delta) {
    await prisma.storage.update({
      where: { ecosystemId },
      data: {
        used: {
          increment: delta
        }
      }
    })
  }
}

Middlewares

Upload Middleware (Multer)

javascript
// middlewares/upload.middleware.js
const multer = require('multer')

// Stockage en mémoire (buffer)
const storage = multer.memoryStorage()

const upload = multer({
  storage: storage,
  limits: {
    fileSize: 50 * 1024 * 1024 // 50 MB
  },
  fileFilter: (req, file, cb) => {
    const allowedTypes = [
      'image/jpeg',
      'image/png',
      'image/gif',
      'image/webp',
      'video/mp4',
      'video/webm',
      'application/pdf'
    ]

    if (allowedTypes.includes(file.mimetype)) {
      cb(null, true)
    } else {
      cb(new ApiError(400, 'Invalid file type'))
    }
  }
})

module.exports = {
  uploadSingle: upload.single('file'),
  uploadMultiple: upload.array('files', 10),
  uploadFields: upload.fields([
    { name: 'image', maxCount: 1 },
    { name: 'thumbnail', maxCount: 1 }
  ])
}

Quota Middleware

javascript
// middlewares/quota.middleware.js
const checkQuota = async (req, res, next) => {
  const { ecosystemId } = req.body
  const fileSize = req.file ? req.file.size : req.body.size

  try {
    await quotaService.checkQuota(ecosystemId, fileSize)
    next()
  } catch (error) {
    res.status(413).json({
      error: 'Storage quota exceeded',
      quota: await quotaService.getQuota(ecosystemId)
    })
  }
}

Événements AMQP

Événements Consommés

Routing KeyPayloadAction
roomee.storage.media{ ecosystemId, max_size, unit }Créer quota storage

Consumer :

javascript
// amqp/storageConsumer.js
amqpServer.subscribe('roomee.storage.media', async (message) => {
  const { ecosystemId, payload } = message

  await prisma.storage.create({
    data: {
      ecosystemId: ecosystemId,
      maxSize: payload.max_size,
      unit: payload.unit,
      used: 0
    }
  })

  logger.info('Storage quota created', { ecosystemId, maxSize: payload.max_size })
})

Configuration

Variables d'Environnement

bash
# Server
PORT=3003
NODE_ENV=development

# Database (minimal)
DATABASE_URL=mongodb://localhost:27017/roomee_media

# JWT
JWT_ACCESS_SECRET=your-jwt-secret

# Google Cloud Storage
GCP_PROJECT_ID=your-gcp-project-id
GCP_BUCKET_NAME=roomee-storage
GCP_KEY_FILE=./config/gcp-service-account-key.json

# Upload Limits
MAX_FILE_SIZE=52428800
MAX_CHUNK_SIZE=1048576

# AMQP
AMQP_GATEWAY_URL=amqp://localhost:5672
AMQP_EXCHANGE_NAME=roomee_events
AMQP_ROUTING_KEY_BASE=roomee.storage

Fichier GCP Service Account

json
{
  "type": "service_account",
  "project_id": "your-project",
  "private_key_id": "...",
  "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
  "client_email": "storage@your-project.iam.gserviceaccount.com",
  "client_id": "...",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token"
}

Organisation GCP Bucket

roomee-storage/
├── ecosystems/
│   ├── {ecosystemId}/
│   │   ├── photos/
│   │   │   ├── photo1.jpg
│   │   │   └── thumbnails/
│   │   │       └── photo1-thumb.jpg
│   │   ├── documents/
│   │   │   └── menu.pdf
│   │   ├── videos/
│   │   │   └── presentation.mp4
│   │   └── avatars/
│   │       └── member-123.jpg
├── qr-codes/
│   └── hotel-{hotelId}.png
└── temp/
    └── chunks/
        └── upload-{uploadId}/

Tests

javascript
// __tests__/media.test.js
describe('MediaController', () => {
  describe('generatePresignedUrl', () => {
    it('should generate a presigned URL', async () => {
      const response = await request(app)
        .post('/media/generate-presigned-url')
        .set('Authorization', `Bearer ${token}`)
        .send({
          filename: 'test.jpg',
          contentType: 'image/jpeg',
          size: 1024000,
          ecosystemId: ecosystemId,
          folder: 'photos'
        })
        .expect(200)

      expect(response.body).toHaveProperty('uploadUrl')
      expect(response.body).toHaveProperty('fileUrl')
      expect(response.body).toHaveProperty('expiresAt')
    })
  })

  describe('uploadFile', () => {
    it('should upload a file to GCP', async () => {
      const response = await request(app)
        .post('/media/upload')
        .set('Authorization', `Bearer ${token}`)
        .attach('file', './test-files/image.jpg')
        .field('ecosystemId', ecosystemId)
        .field('folder', 'photos')
        .expect(200)

      expect(response.body).toHaveProperty('fileUrl')
      expect(response.body).toHaveProperty('thumbnailUrl')
    })
  })
})

Bonnes Pratiques

1. Toujours vérifier le quota avant upload

javascript
await quotaService.checkQuota(ecosystemId, fileSize)

2. Générer des thumbnails automatiquement pour les images

javascript
if (contentType.startsWith('image/')) {
  const thumbnail = await thumbnailService.generateThumbnail(fileBuffer)
  const thumbnailUrl = await gcpService.uploadFile(thumbnailPath, thumbnail)
}

3. Optimiser les images avant upload

javascript
const optimized = await conversionService.optimizeImage(fileBuffer, 'webp')

4. Nettoyer les fichiers temporaires

javascript
// Après upload par chunks
await cleanupTempFiles(uploadId)

5. Utiliser des presigned URLs pour les uploads

javascript
// Meilleur pour la performance
const { uploadUrl } = await generatePresignedUrl()
// Client upload directement vers GCP

Limitations et Améliorations Futures

Limitations Actuelles

  • ❌ Pas de CDN devant GCP Storage
  • ❌ Pas de compression vidéo
  • ❌ Pas de watermarking
  • ❌ Pas de détection de contenu inapproprié

Roadmap

  • [ ] Intégration CDN (Cloudflare/CloudFront)
  • [ ] Compression vidéo (FFmpeg)
  • [ ] Watermarking automatique
  • [ ] AI Content Moderation (Google Vision API)
  • [ ] Multi-région GCP
  • [ ] Support S3 (alternative à GCP)
  • [ ] Galleries et albums
  • [ ] Métadonnées EXIF preservation

Documentation technique Roomee Services